JAVA-240 · Lesson 6 of 12 · 22 min
Executors and thread pools
▶ Video coming soon A video version of this lesson is on the way.
Monday, February 14, 2028. Hamza's latest prototype starts a new thread for every order in the inbox. On a Tuesday with twelve orders it is fine. Ingrid asks what happens on the Saturday Kofi is worried about, with 400 orders, or on the Saturday after a newspaper story about the shop, with 4,000. Four thousand platform threads, each with its own stack, would strain the computer and gain nothing, because the supplier would not answer 4,000 requests at once anyway. And every thread Hamza starts by hand is one he has to join, one whose exceptions he has to catch, one he has to remember to stop. Tonight he learns how professional Java code separates the work to be done from the threads that do it.
The Executor idea
An executor runs tasks you give it. You hand over a Runnable or a Callable, and the executor decides which thread runs it and when. The idea is small but powerful: the code that creates work no longer needs to know anything about threads. You can switch from one worker thread to eight, or to a virtual thread per task, by changing the line that creates the executor and nothing else.
The interface you will use almost all the time is ExecutorService, which adds methods for submitting tasks that return results and for shutting the executor down. The class Executors has factory methods that create the common kinds.
| Factory method | What it creates | Use it for |
|---|---|---|
| newFixedThreadPool(n) | A pool of n reusable platform threads and an unbounded queue | Bounded parallelism, CPU-bound work, limiting load on a service |
| newCachedThreadPool() | A pool that creates threads as needed and reuses idle ones | Many short tasks; risky if tasks pile up |
| newSingleThreadExecutor() | One worker thread with a queue | Tasks that must run one at a time, in order |
| newScheduledThreadPool(n) | A pool that can run tasks after a delay or repeatedly | Refreshing a price list every hour |
| newVirtualThreadPerTaskExecutor() | A new virtual thread for every task (Java 21) | Large numbers of I/O-bound tasks; see lesson 8 |
Thread pools
A thread pool keeps a set of worker threads alive and feeds them tasks from a queue. When a worker finishes one task it takes the next from the queue instead of dying. Reusing threads saves the cost of creating and destroying them, and, just as important, the size of the pool puts a ceiling on how many tasks run at once. A fixed pool of 5 threads will never call the supplier more than 5 times at the same moment, however many orders arrive.
import java.util.*;
import java.util.concurrent.*;
public class Main {
public static void main(String[] args) throws Exception {
ExecutorService pool = Executors.newFixedThreadPool(3, Thread.ofPlatform().name("order-", 1).factory());
Set<String> workerNames = ConcurrentHashMap.newKeySet();
for (int i = 1; i <= 9; i++) {
pool.execute(() -> workerNames.add(Thread.currentThread().getName()));
}
pool.shutdown();
pool.awaitTermination(10, TimeUnit.SECONDS);
System.out.println("tasks run by: " + new TreeSet<>(workerNames));
System.out.println("pool finished: " + pool.isTerminated());
}
}tasks run by: [order-1, order-2, order-3] pool finished: true
What this code does
A fixed pool of three named platform threads runs nine small tasks, and each task records the name of the thread that ran it. However the nine tasks were shared out, only three threads ever did the work, named order-1 to order-3. The program prints the sorted set of names rather than which thread ran which task, because that assignment changes from run to run.
The second argument to newFixedThreadPool is a ThreadFactory, the object a pool asks whenever it needs a new thread. Java 21's builders make one in a single line, which is the easiest way to give pool threads useful names. A pool without a factory names its threads pool-1-thread-1 and so on.
Callable and Future
execute takes a Runnable, which returns nothing. Most real tasks produce a result: a price, a total, a booking. For those, write a Callable, whose call method returns a value and may throw a checked exception, and pass it to submit. submit returns at once with a Future, an object that stands for the result that will exist when the task is finished. Calling get on the future waits until the result is ready and returns it.
import java.util.*;
import java.util.concurrent.*;
public class Main {
static int lookUpCents(String code) throws InterruptedException {
Thread.sleep(100);
return switch (code) {
case "TUBE-700x25" -> 899;
case "PAD-ROAD" -> 1499;
case "CHAIN-11" -> 3999;
default -> 0;
};
}
public static void main(String[] args) throws Exception {
List<String> order = List.of("TUBE-700x25", "TUBE-700x25", "PAD-ROAD", "CHAIN-11");
List<Future<Integer>> prices = new ArrayList<>();
try (ExecutorService pool = Executors.newFixedThreadPool(4)) {
for (String code : order) {
Callable<Integer> lookup = () -> lookUpCents(code);
prices.add(pool.submit(lookup));
}
int total = 0;
for (int i = 0; i < order.size(); i++) {
int cents = prices.get(i).get();
System.out.println(order.get(i) + " " + cents);
total += cents;
}
System.out.println("order total: " + total + " cents");
}
}
}TUBE-700x25 899 TUBE-700x25 899 PAD-ROAD 1499 CHAIN-11 3999 order total: 7296 cents
What this code does
Each part's price lookup is submitted to a pool as a Callable, and the futures are kept in a list in the same order as the order lines. Main then calls get on each future in turn, which waits for that lookup if it is not finished yet. Because main reads the futures in list order, the lines print in order no matter which lookup finished first.
Futures solve the two problems of raw threads from lesson 2. Results come back to the thread that asked for them, without shared arrays. And if the task throws, the exception is not lost: get throws an ExecutionException whose cause is the original exception, so the caller can handle the failure.
import java.util.concurrent.*;
public class Main {
public static void main(String[] args) throws InterruptedException {
try (ExecutorService pool = Executors.newSingleThreadExecutor()) {
Future<Integer> stock = pool.submit(() -> {
String code = "SPOKE-286";
throw new IllegalArgumentException("unknown part " + code);
});
try {
System.out.println("stock: " + stock.get());
} catch (ExecutionException e) {
System.out.println("lookup failed: " + e.getCause().getMessage());
System.out.println("cause type: " + e.getCause().getClass().getSimpleName());
}
}
}
}lookup failed: unknown part SPOKE-286 cause type: IllegalArgumentException
What this code does
The task throws an IllegalArgumentException on the pool's thread. The exception is captured in the future, and get rethrows it wrapped in an ExecutionException on the main thread. The original exception is available through getCause, so main can report exactly what went wrong.
get also has a timed version, get(timeout, unit), which throws TimeoutException if the result is not ready in time. A future can be cancelled with cancel(true), which interrupts the task if it is running. Both are tools for not waiting forever, and lesson 7 uses them.
Submitting many tasks: invokeAll
When you have a batch of tasks and want all their results, invokeAll takes a collection of Callables, runs them, waits until every one has finished, and returns their futures in the same order as the tasks. invokeAny is its partner for when any one result will do: it returns the first successful result and cancels the rest.
import java.util.*;
import java.util.concurrent.*;
public class Main {
record Line(String code, int qty, int cents) {}
public static void main(String[] args) throws Exception {
List<Line> lines = List.of(new Line("TUBE-700x25", 3, 899), new Line("PAD-ROAD", 2, 1499),
new Line("BELL", 1, 1250), new Line("CABLE-BRAKE", 4, 450));
List<Callable<Integer>> subtotals = new ArrayList<>();
for (Line l : lines) subtotals.add(() -> l.qty() * l.cents());
try (ExecutorService pool = Executors.newFixedThreadPool(2)) {
List<Future<Integer>> results = pool.invokeAll(subtotals);
int total = 0;
for (int i = 0; i < lines.size(); i++) {
int sub = results.get(i).get();
System.out.printf("%-12s %5d%n", lines.get(i).code(), sub);
total += sub;
}
System.out.printf("%-12s %5d%n", "TOTAL", total);
}
}
}TUBE-700x25 2697 PAD-ROAD 2998 BELL 1250 CABLE-BRAKE 1800 TOTAL 8745
What this code does
Four line subtotals are computed as a batch with invokeAll on a pool of two threads. invokeAll waits until every task is done and returns the futures in the same order as the tasks, so each get returns at once and the lines print in order. The total is the same on every run because it is added up on the main thread from settled results.
Shutting down
An executor's threads are ordinary non-daemon threads, so a program that forgets to shut its executor down may never exit. There are two methods. shutdown stops the executor from accepting new tasks but lets the queued and running ones finish. shutdownNow also tries to stop running tasks by interrupting them and returns the tasks that never started. After either, awaitTermination waits, up to a time limit, until everything has finished.
Since Java 19, ExecutorService implements AutoCloseable, so a try-with-resources block does the whole job: when the block ends, close calls shutdown and then waits until every task has finished. Most examples in this course use that form. It is the simplest way to be sure that every task has completed, and that every result and exception has been dealt with, before the program moves on.
| Method | Accepts new tasks? | Running tasks | Waits? |
|---|---|---|---|
| shutdown() | No | Allowed to finish | No; call awaitTermination to wait |
| shutdownNow() | No | Interrupted; queued tasks returned unrun | No |
| awaitTermination(t, unit) | Not applicable | Not affected | Yes, up to the time limit |
| close() (Java 19 and later) | No | Allowed to finish | Yes, until all are done |
How big should a pool be?
For CPU-bound work, a pool about the size of the number of cores keeps every core busy without wasting time switching. Runtime.getRuntime().availableProcessors() reports the number the JVM can use. For I/O-bound work the right size is larger, and it is often set by what is on the other end: if the supplier allows 5 connections at once, a pool of 5 for price checks makes sense whatever the number of cores. A rough rule from Brian Goetz's book Java Concurrency in Practice is threads = cores × (1 + wait time ÷ compute time). With 4 cores and tasks that wait 900 milliseconds for every 100 they compute, that gives 4 × (1 + 9) = 40 threads. For large numbers of waiting tasks, lesson 8's virtual threads often make the calculation unnecessary.
There is one more choice inside every pool: its queue. The fixed and single-thread pools use an unbounded queue, so they never refuse a task, but a flood of tasks can fill memory. A ThreadPoolExecutor built directly can use a bounded queue and a rejection policy that decides what happens when the queue is full, such as throwing RejectedExecutionException or making the submitting thread run the task itself, which naturally slows down a producer that is going too fast.
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicInteger;
public class Main {
public static void main(String[] args) throws InterruptedException {
AtomicInteger done = new AtomicInteger();
ThreadPoolExecutor pool = new ThreadPoolExecutor(
2, 2, 0, TimeUnit.SECONDS,
new ArrayBlockingQueue<>(4),
new ThreadPoolExecutor.CallerRunsPolicy());
for (int i = 0; i < 20; i++) {
pool.execute(() -> {
try { Thread.sleep(20); } catch (InterruptedException e) { Thread.currentThread().interrupt(); }
done.incrementAndGet();
});
}
pool.shutdown();
pool.awaitTermination(10, TimeUnit.SECONDS);
System.out.println("tasks completed: " + done.get());
System.out.println("largest number of pool threads: " + pool.getLargestPoolSize());
}
}tasks completed: 20 largest number of pool threads: 2
What this code does
This pool has exactly two threads and a queue that holds only four waiting tasks. When the queue is full, the CallerRunsPolicy makes the main thread run the extra task itself, which slows main down until the workers catch up. All twenty tasks complete, nothing is rejected, and the pool never grows beyond two threads.
Scheduled tasks
A ScheduledExecutorService runs tasks after a delay or on a fixed schedule. OrderDesk will use one to refresh its copy of the supplier's price list every hour while it runs. scheduleAtFixedRate starts a task at regular intervals measured from start to start; scheduleWithFixedDelay waits a fixed time after each run ends before starting the next. If a scheduled task throws, later runs are silently cancelled, so wrap the body of a repeating task in a try and catch that logs the problem.
try (ScheduledExecutorService timer = Executors.newSingleThreadScheduledExecutor()) {
timer.scheduleAtFixedRate(() -> {
try {
priceList.refreshFromSupplier();
} catch (Exception e) {
log.add("price refresh failed: " + e.getMessage());
}
}, 0, 1, TimeUnit.HOURS);
runTheSaturdayInbox();
}What this code does
The timer refreshes the price list at once and then every hour while the Saturday inbox is processed. The try and catch inside the task keep one failed refresh from cancelling all the later ones. When the try-with-resources block ends, close shuts the timer down, so no refresh runs after the program has finished its work.
Try it yourself
Prices in parallel
Use invokeAll on a fixed pool of 3 threads to compute the price of each repair: a repair's price is hours × 6500 cents. Print each repair and its price in order, then the total.
The program should print tune-up 6500, wheel build 19500, brake bleed 9750 and total 35750.
import java.util.*;
import java.util.concurrent.*;
public class Main {
record Repair(String name, double hours) {}
public static void main(String[] args) throws Exception {
List<Repair> jobs = List.of(new Repair("tune-up", 1.0), new Repair("wheel build", 3.0), new Repair("brake bleed", 1.5));
int total = 0;
// build a list of Callables, run them with invokeAll on 3 threads, print each price
System.out.println("total " + total);
}
}Try it yourself
Catch the failed lookup
Submit a Callable that throws IllegalStateException("supplier offline") and catch the ExecutionException that get throws. Print the cause's message, then print still running: false after the try-with-resources block has closed the pool.
The program should print failed: supplier offline and still running: false.
import java.util.concurrent.*;
public class Main {
public static void main(String[] args) throws InterruptedException {
ExecutorService pool = Executors.newFixedThreadPool(1);
Future<Integer> f = pool.submit(() -> 42);
try {
System.out.println("price: " + f.get());
} catch (ExecutionException e) {
System.out.println("failed: " + e.getCause().getMessage());
}
System.out.println("still running: " + !pool.isTerminated());
pool.shutdownNow();
}
}Common mistakes
MistakeStarting a new platform thread for every task.
InsteadSubmit tasks to an executor and let its pool size limit how many run at once.
MistakeForgetting to shut an executor down.
InsteadUse try-with-resources (Java 19 and later) or call shutdown and awaitTermination.
MistakeIgnoring the Future returned by submit.
InsteadCall get on it, or exceptions thrown by the task are silently kept inside the future.
MistakeCalling get immediately after each submit.
InsteadSubmit all the tasks first, then get the results, or the tasks run one at a time.
MistakeLetting a repeating scheduled task throw.
InsteadCatch and log exceptions inside the task; an uncaught one cancels all later runs.
Summary and key terms
- An executor runs submitted tasks and hides the threads that run them; ExecutorService adds submit, invokeAll and shutdown.
- Thread pools reuse a fixed set of worker threads fed from a queue, which saves thread creation and limits how many tasks run at once.
- A ThreadFactory, such as Thread.ofPlatform().name("order-", 1).factory(), names pool threads.
- A Callable returns a value and may throw; submit returns a Future whose get waits for the result and rethrows any exception wrapped in ExecutionException. invokeAll runs a batch and returns futures in task order, and invokeAny returns the first successful result. shutdown stops new tasks, shutdownNow also interrupts running ones, and since Java 19 try-with-resources closes and waits.
- Size pools by the kind of work: about one thread per core for CPU-bound tasks, more for I/O-bound tasks or the limit of the service being called.
- ThreadPoolExecutor lets you choose a bounded queue and a rejection policy such as CallerRunsPolicy.
- Scheduled executors run tasks after delays or repeatedly, and repeating tasks must catch their own exceptions.
- Executor
- An object that runs submitted tasks, choosing which thread runs each.
- ExecutorService
- An executor that can also return futures and be shut down.
- Thread pool
- A set of reusable worker threads that take tasks from a queue.
- ThreadFactory
- An object a pool calls to create each new thread.
- Callable
- A task that returns a value and may throw a checked exception.
- Future
- An object representing the result of a task that may not have finished yet.
- ExecutionException
- The exception get throws when the task failed; its cause is the original exception.
- invokeAll
- A method that runs a collection of tasks and returns their futures in order when all are done.
- shutdown
- A method that stops an executor accepting new tasks while letting existing ones finish.
- Rejection policy
- What a ThreadPoolExecutor does with a task when its queue is full.
- CallerRunsPolicy
- A rejection policy that makes the submitting thread run the task itself.
- ScheduledExecutorService
- An executor that can run tasks after a delay or on a repeating schedule.
Practice problems
Work each problem on paper first, then reveal the answer to check.
What does submit return?
Show answer
A Future for the task's result.
A task throws NullPointerException. What does get throw?
Show answer
ExecutionException, with the NullPointerException as its cause.
What does close do on an ExecutorService in Java 21?
Show answer
It calls shutdown and waits until all submitted tasks have finished.
Using the rough rule, how many threads suit 8 cores and tasks that wait 4 times as long as they compute?
Show answer
8 × (1 + 4) = 40.
Why print the sorted set of pool thread names instead of which thread ran which task?
Show answer
The assignment of tasks to threads changes from run to run; the set of names does not.
Lesson quiz · 5 questions
Score 60% or more to complete this lesson.
Create a free account to save your progress, take quizzes and earn a certificate.