JAVA-240 · Lesson 10 of 12 · 23 min
Producer-consumer patterns
▶ Video coming soon A video version of this lesson is on the way.
Monday, March 20, 2028. OrderDesk has taken shape on the whiteboard. One part of the program reads the inbox file and turns each line into an order. Several workers check prices and reserve stock. Another part writes the picking list and updates the counter screen. Each part works at its own speed: reading the file is fast, price checks are slow, printing is fast again. Ingrid draws arrows between the boxes and writes one word on each arrow: queue. Tonight's pattern, producers and consumers connected by a queue, is probably the single most useful design in concurrent programming.
The pattern
In the producer-consumer pattern, one or more producer threads create pieces of work and put them into a shared queue, and one or more consumer threads take the work out and process it. The producers and consumers never talk to each other directly; they only know about the queue. That separation is the point. Producers do not need to know how many consumers there are or how fast they work, and consumers do not need to know where the work came from. You can add consumers to go faster, or replace the producer with one that reads from a network instead of a file, without changing the other side.
The queue must be thread-safe, and it should make threads wait rather than spin. That is exactly what the BlockingQueue implementations of lesson 5 provide. put adds an item, waiting if the queue is full; take removes one, waiting if the queue is empty. A waiting thread uses no processor time, and a virtual thread waiting in put or take unmounts from its carrier.
A first pipeline
The program below has one producer, which reads eight inbox lines, and one consumer, which picks them. The queue holds at most three orders at a time, so the producer has to wait whenever the consumer falls behind. With a single consumer, the orders are picked in exactly the order they were read, which is the property that makes this program's output the same every time.
import java.util.*;
import java.util.concurrent.*;
public class Main {
record Order(int id, String part, int qty) {}
public static void main(String[] args) throws InterruptedException {
List<String> inbox = List.of("1041,TUBE-700x25,2", "1042,PAD-ROAD,1", "1043,CHAIN-11,1", "1044,BELL,3",
"1045,TUBE-700x25,1", "1046,CABLE-BRAKE,4", "1047,PAD-ROAD,2", "1048,BELL,1");
BlockingQueue<Order> toPick = new ArrayBlockingQueue<>(3);
List<String> picked = new ArrayList<>();
Thread reader = new Thread(() -> {
try {
for (String line : inbox) {
String[] f = line.split(",");
toPick.put(new Order(Integer.parseInt(f[0]), f[1], Integer.parseInt(f[2])));
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}, "inbox-reader");
Thread picker = new Thread(() -> {
try {
for (int i = 0; i < inbox.size(); i++) {
Order o = toPick.take();
picked.add(o.id() + ": " + o.qty() + " x " + o.part());
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}, "picker");
reader.start();
picker.start();
reader.join();
picker.join();
picked.forEach(System.out::println);
}
}1041: 2 x TUBE-700x25 1042: 1 x PAD-ROAD 1043: 1 x CHAIN-11 1044: 3 x BELL 1045: 1 x TUBE-700x25 1046: 4 x CABLE-BRAKE 1047: 2 x PAD-ROAD 1048: 1 x BELL
What this code does
The inbox reader turns each line into an Order record and puts it in a queue with room for three, waiting whenever the queue is full. The picker takes orders one at a time, waiting whenever the queue is empty, and records them. Because there is one producer, one consumer and a first-in, first-out queue, the orders always come out in the order they were read; main prints them after joining both threads.
The consumer in that program knows in advance how many orders to expect. Real consumers usually do not: the inbox might have 12 lines or 400. They need another way to learn that the work is finished.
Poison pills
A common way to tell consumers to stop is a poison pill: a special item that means no more work is coming. The producer puts one pill into the queue for every consumer after the last real item. Each consumer, when it takes a pill, stops. Because the queue is first-in, first-out, a consumer only reaches a pill after every real item put before it has been taken by someone.
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicInteger;
public class Main {
record Order(int id, int cents) {}
static final Order PILL = new Order(-1, 0);
public static void main(String[] args) throws InterruptedException {
int workers = 3;
BlockingQueue<Order> queue = new LinkedBlockingQueue<>(10);
AtomicInteger orders = new AtomicInteger();
AtomicInteger cents = new AtomicInteger();
Set<Integer> seen = ConcurrentHashMap.newKeySet();
try (ExecutorService pool = Executors.newFixedThreadPool(workers)) {
for (int w = 0; w < workers; w++) {
pool.submit(() -> {
while (true) {
Order o = queue.take();
if (o == PILL) return null;
Thread.sleep(5);
orders.incrementAndGet();
cents.addAndGet(o.cents());
seen.add(o.id());
}
});
}
for (int id = 1; id <= 40; id++) queue.put(new Order(id, 500 + id));
for (int w = 0; w < workers; w++) queue.put(PILL);
}
System.out.println("orders processed: " + orders.get());
System.out.println("distinct orders: " + seen.size());
System.out.println("total: " + cents.get() + " cents");
}
}orders processed: 40 distinct orders: 40 total: 20820 cents
What this code does
Three workers in a pool take orders from a bounded queue until each takes a poison pill. Main produces forty orders and then one pill per worker; the try-with-resources block waits until all three workers have stopped. Which worker handled which order varies, so the program prints only totals, which are the same every run: forty orders, each counted once.
The pill is compared with ==, identity, not equals. Using one shared PILL object means no real order can ever be mistaken for it, even one that happens to have the same field values. With several consumers, there must be exactly one pill per consumer, or some consumers will wait forever on an empty queue, and the pool will never close.
Backpressure
A bounded queue gives the system backpressure: when consumers fall behind, the queue fills up and producers are slowed down by put, instead of piling up unbounded work in memory. An unbounded queue, such as new LinkedBlockingQueue() with no capacity, never pushes back. That is fine when the producer is naturally limited, such as one file of a few hundred lines, and dangerous when it is not, such as a web form during a sale.
Sometimes a producer would rather not wait. offer adds an item if there is room and returns false at once if there is not; offer with a time limit waits up to that long. poll is the consumer's equivalent: it returns null if nothing arrives in time. These let a program decide what to do under load: tell the customer to try again, drop a low-value item, or log a warning.
import java.util.concurrent.*;
public class Main {
public static void main(String[] args) throws InterruptedException {
BlockingQueue<String> screenUpdates = new ArrayBlockingQueue<>(2);
System.out.println("offer 1: " + screenUpdates.offer("order 1041 ready"));
System.out.println("offer 2: " + screenUpdates.offer("order 1042 ready"));
System.out.println("offer 3: " + screenUpdates.offer("order 1043 ready"));
System.out.println("offer 4, waiting up to 100 ms: "
+ screenUpdates.offer("order 1044 ready", 100, TimeUnit.MILLISECONDS));
System.out.println("take: " + screenUpdates.take());
System.out.println("offer 5: " + screenUpdates.offer("order 1045 ready"));
System.out.println("in the queue: " + screenUpdates);
screenUpdates.clear();
System.out.println("poll on empty queue, waiting up to 50 ms: " + screenUpdates.poll(50, TimeUnit.MILLISECONDS));
}
}offer 1: true offer 2: true offer 3: false offer 4, waiting up to 100 ms: false take: order 1041 ready offer 5: true in the queue: [order 1042 ready, order 1045 ready] poll on empty queue, waiting up to 50 ms: null
What this code does
A queue with room for two accepts two updates and then refuses a third with false, without waiting; a timed offer waits 100 milliseconds and also gives up. After one item is taken, there is room again and the next offer succeeds. A timed poll on an empty queue returns null when its time is up.
| Operation | Throws if impossible | Returns a special value | Waits | Waits up to a limit |
|---|---|---|---|---|
| Add | add(e) | offer(e) gives false | put(e) | offer(e, time, unit) |
| Remove | remove() | poll() gives null | take() | poll(time, unit) |
| Look at the head | element() | peek() gives null | not applicable | not applicable |
Several stages
A pipeline chains producer-consumer pairs: the consumer of one queue is the producer of the next. OrderDesk's design has three stages: read the inbox, price and reserve, and write the picking list. Each stage can have as many threads as suits its work. Reading needs one thread, pricing needs many because it waits on the supplier, and writing the list needs one so that lines are never interleaved.
import java.util.*;
import java.util.concurrent.*;
public class Main {
record Order(int id, String part) {}
record Priced(int id, String part, int cents) {}
static final Order STOP = new Order(-1, "");
static final Priced DONE = new Priced(-1, "", 0);
static int price(String part) throws InterruptedException {
Thread.sleep(20);
return switch (part) { case "TUBE" -> 899; case "PAD" -> 1499; default -> 1250; };
}
public static void main(String[] args) throws Exception {
BlockingQueue<Order> toPrice = new LinkedBlockingQueue<>(5);
BlockingQueue<Priced> toWrite = new LinkedBlockingQueue<>(5);
int pricers = 4;
List<Priced> written = new ArrayList<>();
try (ExecutorService ex = Executors.newVirtualThreadPerTaskExecutor()) {
for (int p = 0; p < pricers; p++) {
ex.submit(() -> {
for (Order o = toPrice.take(); o != STOP; o = toPrice.take())
toWrite.put(new Priced(o.id(), o.part(), price(o.part())));
toWrite.put(DONE);
return null;
});
}
ex.submit(() -> {
int finished = 0;
while (finished < pricers) {
Priced p = toWrite.take();
if (p == DONE) finished++;
else written.add(p);
}
return null;
});
String[] parts = {"TUBE", "PAD", "BELL"};
for (int id = 1; id <= 12; id++) toPrice.put(new Order(1040 + id, parts[id % 3]));
for (int p = 0; p < pricers; p++) toPrice.put(STOP);
}
written.sort(Comparator.comparingInt(Priced::id));
for (Priced p : written) System.out.println(p.id() + " " + p.part() + " " + p.cents());
}
}1041 PAD 1499 1042 BELL 1250 1043 TUBE 899 1044 PAD 1499 1045 BELL 1250 1046 TUBE 899 1047 PAD 1499 1048 BELL 1250 1049 TUBE 899 1050 PAD 1499 1051 BELL 1250 1052 TUBE 899
What this code does
Main reads twelve orders into the first queue, four pricing threads move them to the second queue with a price, and one writer collects them. Each pricer sends a DONE marker when it stops, and the writer stops after seeing all four, so nothing is lost at the end. The four pricers finish in an unpredictable order, so the picking list is sorted by order number before it is printed.
That example shows how pills multiply through a pipeline: each stage must tell the next stage that it has finished, and a stage with several threads is finished only when all of them are. Getting this right is fiddly, which is one reason Ingrid suggests using an executor's own shutdown, or CompletableFuture's allOf, where they fit, and saving explicit queues for real streams of work.
Waiting for a group: CountDownLatch
A CountDownLatch starts with a count. Threads call countDown when they finish something, and any thread that calls await waits until the count reaches zero. It is a one-shot gate: once open, it stays open. OrderDesk uses one at startup, so that the order workers do not begin until the stock table and the price list have both loaded.
import java.util.*;
import java.util.concurrent.*;
public class Main {
public static void main(String[] args) throws InterruptedException {
CountDownLatch ready = new CountDownLatch(2);
Map<String, Integer> stock = new ConcurrentHashMap<>();
Map<String, Integer> prices = new ConcurrentHashMap<>();
Thread.startVirtualThread(() -> {
stock.putAll(Map.of("TUBE-700x25", 14, "BELL", 3));
ready.countDown();
});
Thread.startVirtualThread(() -> {
prices.putAll(Map.of("TUBE-700x25", 899, "BELL", 1250));
ready.countDown();
});
boolean loaded = ready.await(5, TimeUnit.SECONDS);
System.out.println("both tables loaded: " + loaded);
System.out.println("stock entries: " + stock.size() + ", price entries: " + prices.size());
System.out.println("BELL: " + stock.get("BELL") + " at " + prices.get("BELL") + " cents");
}
}both tables loaded: true stock entries: 2, price entries: 2 BELL: 3 at 1250 cents
What this code does
Two loader threads fill the stock table and the price list, and each counts the latch down once when it is done. Main waits on the latch, with a five-second limit for safety, before using either table. await also makes the loaders' writes visible to main, because counting down happens-before a successful await returns.
The java.util.concurrent package has a few more coordinators of this kind. A CyclicBarrier makes a fixed number of threads wait for each other at a point and can be reused round after round. A Phaser is a more flexible barrier whose number of parties can change. An Exchanger lets two threads swap objects. They are less common than queues and latches, but worth recognizing when you meet them in existing code.
The older way: wait and notify
Before java.util.concurrent existed, producer-consumer queues were written by hand with synchronized, wait and notifyAll. A consumer holding the queue's monitor calls wait, which releases the monitor and suspends the thread until another thread calls notifyAll on the same object. You will still see this in older code and in interview questions, so it is worth reading once:
class Tray {
private final java.util.LinkedList<String> items = new java.util.LinkedList<>();
private final int capacity = 3;
synchronized void put(String item) throws InterruptedException {
while (items.size() == capacity) wait();
items.addLast(item);
notifyAll();
}
synchronized String take() throws InterruptedException {
while (items.isEmpty()) wait();
String item = items.removeFirst();
notifyAll();
return item;
}
}What this code does
put waits while the tray is full and take waits while it is empty, each inside a while loop that rechecks the condition after waking. After changing the tray, each method calls notifyAll so that waiting threads wake and check again. ArrayBlockingQueue does the same job, tested and with timed versions, which is why new code should use it instead.
The while loops are essential. A waiting thread can wake up even though nobody notified it, which is called a spurious wakeup, and another thread may have taken the item in the moment between the notify and the wake-up. Checking the condition in a loop handles both.
Try it yourself
Three pickers and their pills
Three consumer threads take numbers from a LinkedBlockingQueue and add them to a shared AtomicLong until each takes the poison pill -1. Main puts the numbers 1 to 1,000 and then one pill per consumer, and waits for the consumers by closing the pool.
The program should print consumers stopped: 3 and sum: 500500.
import java.util.concurrent.*;
import java.util.concurrent.atomic.*;
public class Main {
public static void main(String[] args) throws InterruptedException {
BlockingQueue<Integer> queue = new LinkedBlockingQueue<>(50);
AtomicLong sum = new AtomicLong();
AtomicInteger stopped = new AtomicInteger();
try (ExecutorService pool = Executors.newFixedThreadPool(3)) {
// start three consumers, then put 1..1000 and three pills
}
System.out.println("consumers stopped: " + stopped.get());
System.out.println("sum: " + sum.get());
}
}Try it yourself
A full tray says no
A queue for the counter screen holds 3 updates. Offer 5 updates without waiting and count how many were accepted, then take one and offer one more.
The program should print accepted: 3, refused: 2 and after one take, offer gives true.
import java.util.concurrent.*;
public class Main {
public static void main(String[] args) throws InterruptedException {
BlockingQueue<String> screen = new ArrayBlockingQueue<>(3);
int accepted = 0, refused = 0;
// offer "update 1" to "update 5"
System.out.println("accepted: " + accepted);
System.out.println("refused: " + refused);
System.out.println("after one take, offer gives " + false);
}
}Common mistakes
MistakeUsing an unbounded queue between a fast producer and slow consumers.
InsteadUse a bounded queue so put pushes back, or decide explicitly what to do when offer fails.
MistakePutting one poison pill for several consumers.
InsteadPut one pill per consumer, after the last real item.
MistakeComparing a poison pill with equals on a record.
InsteadUse one shared pill object and compare with ==, so no real item can match it.
MistakeSpinning on poll() with no timeout while waiting for work.
InsteadUse take, or poll with a time limit, so the thread waits without burning the processor.
MistakeCalling wait outside a while loop.
InsteadAlways wait in a loop that rechecks the condition, because of spurious wakeups and competing threads.
Summary and key terms
- In the producer-consumer pattern, producers put work into a thread-safe queue and consumers take it out, and neither knows about the other.
- BlockingQueue's put waits when the queue is full and take waits when it is empty, without using the processor.
- With one producer, one consumer and a first-in, first-out queue, items come out in order.
- A poison pill, one per consumer and compared by identity, tells consumers to stop.
- A bounded queue gives backpressure, slowing producers when consumers fall behind; offer and poll, with or without a time limit, let threads decide what to do instead of waiting.
- A pipeline chains several queues, and each stage must tell the next when all of its threads have finished.
- A CountDownLatch lets threads wait until a count of events reaches zero, and also makes the earlier writes visible.
- Hand-made buffers with wait and notifyAll must wait in a while loop, but blocking queues should be used instead.
- Producer-consumer
- A design in which producers put work into a shared queue and consumers take it out.
- Producer
- A thread that creates work items and puts them into a queue.
- Consumer
- A thread that takes work items from a queue and processes them.
- Poison pill
- A special item that tells a consumer no more work is coming.
- Backpressure
- Slowing producers down when consumers cannot keep up, for example with a full bounded queue.
- offer
- A queue method that adds an item only if there is room, returning true or false.
- poll
- A queue method that removes an item if one is available, returning null otherwise.
- Pipeline
- A chain of stages connected by queues, each stage consuming from one queue and producing into the next.
- CountDownLatch
- A one-shot gate that opens when its count has been counted down to zero.
- CyclicBarrier
- A reusable point where a fixed number of threads wait for each other.
- Spurious wakeup
- A thread returning from wait without having been notified.
Practice problems
Work each problem on paper first, then reveal the answer to check.
What does put do on a full ArrayBlockingQueue?
Show answer
It waits until there is room.
Four consumers share a queue. How many poison pills does the producer put?
Show answer
Four, one per consumer.
Why does the two-thread picking example print the orders in order every time?
Show answer
There is one producer, one consumer and a first-in, first-out queue.
What does offer return when the queue is full?
Show answer
false, at once.
Why use a CountDownLatch at OrderDesk's startup?
Show answer
So the workers wait until both the stock table and the price list have loaded.
Lesson quiz · 5 questions
Score 60% or more to complete this lesson.
Create a free account to save your progress, take quizzes and earn a certificate.