Skip to content
javahacks.dev
Concurrency

Java ExecutorService Usage and Thread Pools Done Properly

Use ExecutorService correctly: choose a pool type, submit Runnable and Callable, handle Future results and exceptions, bound the queue and shut down cleanly.

7 min read
Summarize in AI
Flat illustration of threads sharing work across lanes, a lock and a pool, with the OpenJDK, Linux and IntelliJ IDEA logos.

Starting a thread per task works for a demo and fails under load: each platform thread costs memory, and thousands of them thrash the scheduler. A pool fixes both. This works on JDK 17 and later, with notes on JDK 21 and above.

The basics

java
ExecutorService executor = Executors.newFixedThreadPool(4);

Future<Integer> future = executor.submit(() -> expensiveCount());     // Callable<Integer>
executor.execute(() -> log.info("fire and forget"));                  // Runnable

try {
    int count = future.get(2, TimeUnit.SECONDS);
} catch (ExecutionException e) {
    throw new IllegalStateException("task failed", e.getCause());
} catch (TimeoutException e) {
    future.cancel(true);
}

submit accepts a Callable, which returns a value and may throw, or a Runnable. get blocks until the task completes. An exception in the task is wrapped in ExecutionException, with the real cause inside. With execute, an uncaught exception goes to the thread's uncaught handler, and nobody else sees it.

Pool types and their traps

FactoryBehaviourRisk
newFixedThreadPool(n)n threads, unbounded queueQueue grows without limit
newCachedThreadPool()Creates threads as neededUnbounded thread count
newSingleThreadExecutor()One thread, orderedUnbounded queue
newScheduledThreadPool(n)Delayed and periodic tasksTasks that throw stop the schedule
newVirtualThreadPerTaskExecutor()A virtual thread per taskNot a pool, no cap on concurrency

The fixed and single pools use an unbounded LinkedBlockingQueue. A producer faster than the consumers fills memory instead of failing, which ends in an OutOfMemoryError. The cached pool does the opposite: it makes a new thread for each task when none is idle.

A bounded pool

Build the executor directly when you need limits:

java
ThreadPoolExecutor pool = new ThreadPoolExecutor(
        4, 8,                                   // core and maximum threads
        60, TimeUnit.SECONDS,                   // idle time for extra threads
        new ArrayBlockingQueue<>(100),          // bounded queue
        new NamedThreadFactory("orders"),       // names for diagnostics
        new ThreadPoolExecutor.CallerRunsPolicy());

Tasks go to core threads first, then into the queue, and extra threads start only when the queue is full. When both the queue and the maximum are exhausted, the rejection policy applies. AbortPolicy throws RejectedExecutionException, the one in fixing it, and CallerRunsPolicy makes the submitting thread run the task, which slows producers down naturally. Name your threads, see ThreadFactory.

Choosing a size

CPU-bound work needs about one thread per core. Blocking I/O work needs more, depending on the wait ratio, a topic of pool sizing.

Shut down

java
executor.shutdown();                                    // no new tasks, finish queued ones
if (!executor.awaitTermination(30, TimeUnit.SECONDS)) {
    executor.shutdownNow();                              // interrupt running tasks
}

A pool's non-daemon threads keep the JVM alive after main ends. On JDK 19 and later, ExecutorService implements AutoCloseable, so try-with-resources waits for tasks and closes it. A leak is the subject of shutting down without leaks.

Virtual threads

On Java 21 and later, for tasks that mostly wait on I/O:

java
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    List<Future<String>> results = urls.stream().map(u -> executor.submit(() -> fetch(u))).toList();
}

There is no pool to size, and the number of concurrent tasks is limited by the resources they use. Limit calls to a downstream service with a semaphore.

See what a full pool does

A pool has three limits that interact: the number of threads, the size of the queue and what happens when both are full. The documentation describes the rules, and a short program shows them. This pool has two threads, a queue of two, and the default AbortPolicy. Five tasks are submitted while the first two block.

Pools.javajava
ThreadPoolExecutor pool = new ThreadPoolExecutor(2, 2, 0, TimeUnit.SECONDS,
        new ArrayBlockingQueue<>(2), new ThreadPoolExecutor.AbortPolicy());
CountDownLatch release = new CountDownLatch(1);
for (int i = 1; i <= 5; i++) {
    final int id = i;
    try {
        pool.execute(() -> { try { release.await(); } catch (InterruptedException e) { } });
        System.out.println("task " + id + " accepted");
    } catch (RejectedExecutionException e) {
        System.out.println("task " + id + " rejected: " + e.getMessage());
    }
}
Text
task 1 accepted
task 2 accepted
task 3 accepted
task 4 accepted
task 5 rejected: Task Pools$$Lambda$222/0x0000000800d48a08@... rejected from java.util.concurrent.ThreadPoolExecutor@...[Running, pool size = 2, active threads = 2, queued tasks = 2, completed tasks = 0]

Tasks 1 and 2 got threads, tasks 3 and 4 went to the queue, and task 5 found no capacity. The exception message tells you everything needed: the pool is Running, with two active threads and two queued tasks. Capacity is the number of threads plus the queue length, four here, and the fifth submission is where the failure shows. The order matters: a pool grows from its core size to its maximum only after the queue is full, so a pool with a large queue and a larger maximum never reaches the maximum. That is why newFixedThreadPool, which uses an unbounded queue, never rejects anything and instead grows its backlog without limit until the heap is gone.

Choose the rejection policy deliberately

When capacity runs out you must decide whose problem it becomes. The four built-in policies give four answers, and CallerRunsPolicy is the only one that slows the producer down.

java
ThreadPoolExecutor caller = new ThreadPoolExecutor(1, 1, 0, TimeUnit.SECONDS,
        new ArrayBlockingQueue<>(1), new ThreadPoolExecutor.CallerRunsPolicy());
for (int i = 1; i <= 4; i++) {
    final int id = i;
    caller.execute(() -> System.out.println("task " + id + " ran on " + Thread.currentThread().getName()));
}
Text
task 3 ran on main
task 1 ran on pool-2-thread-1
task 2 ran on pool-2-thread-1
task 4 ran on pool-2-thread-1

Task 3 arrived when the thread and the queue were both occupied, so the submitting thread ran it itself. While it was busy running that task, it could not submit more, which throttles the producer to the speed of the consumers. This is backpressure for free, and for a request handler pushing work to a pool it turns an overload into slower responses instead of an exception. Do not use it from a thread that must stay responsive, such as an event loop, because the caller blocks.

PolicyBehaviourUse when
AbortPolicythrows RejectedExecutionExceptioncallers can handle failure and you want to see overload
CallerRunsPolicycaller runs the taskwork must not be dropped and slowing down is acceptable
DiscardPolicydrops silentlyalmost never, since the loss is invisible
DiscardOldestPolicydrops the oldest queued taskonly the freshest result matters, such as a status refresh

Name the threads and watch the pool

An unnamed pool shows up as pool-3-thread-7 in a thread dump, which tells nobody anything. Give the factory a name, and expose the pool's counters as metrics so saturation is visible before it hurts.

java
ThreadFactory names = Thread.ofPlatform().name("invoice-worker-", 1).factory();
ThreadPoolExecutor invoices = new ThreadPoolExecutor(4, 4, 0, TimeUnit.SECONDS,
        new ArrayBlockingQueue<>(100), names, new ThreadPoolExecutor.CallerRunsPolicy());

// every 10 seconds, or as gauges in Micrometer
System.out.printf("active=%d queued=%d completed=%d%n",
        invoices.getActiveCount(), invoices.getQueue().size(), invoices.getCompletedTaskCount());

The numbers to alert on are the queue length approaching its bound and the active count equal to the pool size for several minutes. Either means that producers are faster than consumers, and the fix is a change in rate or capacity, not a bigger queue.

Exceptions vanish with submit

execute lets an uncaught exception reach the thread's handler. submit wraps the task in a FutureTask, which stores the exception inside the Future; if nobody calls get(), the failure is silent. A scheduled job that throws once and is never looked at stops running without any log line. Either call get(), or catch and log inside the task.

java
pool.submit(() -> {
    try {
        invoiceService.sendReminders();
    } catch (Exception e) {
        log.error("reminder run failed", e);      // otherwise the Future swallows it
    }
});

Size the pool from the work, then verify it

A pool size is a hypothesis about the workload. For tasks that wait on I/O most of the time, a pool larger than the core count is right; for tasks that compute, a pool near the core count is right. A starting formula is cores times (1 plus wait time divided by compute time), and the number to trust is the one you measure. Run the real workload with two or three sizes, record throughput and the queue length, and pick the smallest size after which throughput stops improving. Record the choice and the reasoning in a comment, since the next engineer will be tempted to double it when something is slow.

If a pool is small and the queue is long, the system is saturated, and a larger pool helps only if the tasks are waiting. If the pool is large and the CPU is at 100 percent, a larger pool only adds context switching. The metrics from the monitoring section tell you which case you are in.

Where this does not apply

For async pipelines with composition, use CompletableFuture. In an application server or Spring, use the managed or framework executor, as in Jakarta Concurrency. The reference is the [ExecutorService Javadoc](https://docs.oracle.com/en/java/javase/25/docs/api/java.base/java/util/concurrent/ExecutorService.html), and the [ThreadPoolExecutor Javadoc](https://docs.oracle.com/en/java/javase/25/docs/api/java.base/java/util/concurrent/ThreadPoolExecutor.html) explains queuing.

Share

Keep reading

Get new posts by email

We send one email when a new post is published. No spam.

Unsubscribe in one click. No spam, ever.