Thread Pools: ExecutorService

← Back to the section

Creating a new thread for every task is expensive and dangerous. ExecutorService solves this: it keeps ready threads on hand and distributes work among them.

The problem: why you need a pool at all

A thread in Java is an expensive object. Creating one takes on the order of a millisecond and requires allocating memory for the stack (512 KB–1 MB by default). If an application creates a thread for every HTTP request or background task, under load this leads to two problems:

• Memory overuse. Every thread gets its own stack, about a megabyte by default: a thousand threads reserve a gigabyte of address space (less is actually committed — pages are touched lazily).
• Scheduler overload. The operating system keeps switching between thousands of threads, leaving less time for useful work — and this usually hurts before memory does.

A thread pool is a set of pre-created threads that wait for tasks in a queue. A task arrives → a free thread picks it up → runs it → returns to wait for the next one. A thread is created once, not on every request.

ExecutorService: the base interface

ExecutorService is the main interface for managing a pool. It extends Executor, adding the ability to submit tasks that return a result and to manage the lifecycle.

Two ways to hand off a task:

live example

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;

public class PoolBasics {
    public static void main(String[] args) throws Exception {
        ExecutorService pool = Executors.newFixedThreadPool(2);

        pool.execute(() -> System.out.println("execute: no result, thread " + Thread.currentThread().getName()));

        Future<Integer> receipt = pool.submit(() -> {
            Thread.sleep(200);
            return 42;
        });
        System.out.println("receipt in hand, still running: isDone=" + receipt.isDone());
        System.out.println("get() returned: " + receipt.get());

        pool.shutdown();
        System.out.println("stopped within a second: " + pool.awaitTermination(1, TimeUnit.SECONDS));
    }
}

Callable<V> differs from Runnable: it returns a value and may throw a checked exception. Future<V> is the receipt — the task is still running, and future.get() blocks the calling thread until the result is ready.

The Executors factories and their limits

The Executors class provides ready-made factories:

Why an explicit ThreadPoolExecutor is better in production. newFixedThreadPool queues without a limit (LinkedBlockingQueue): tasks arriving faster than they are processed grow the queue until memory runs out. newCachedThreadPool has no cap on threads — one load spike can create thousands.

ThreadPoolExecutor: full control

The same pool, with the dangerous defaults spelled out as constructor arguments. The pool below is deliberately tiny, so all four outcomes fit into four tasks.

live example

import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

public class PoolRouting {
    public static void main(String[] args) throws Exception {
        ThreadPoolExecutor pool = new ThreadPoolExecutor(
                1, 2,                          // corePoolSize — permanent threads, maximumPoolSize — the cap
                30, TimeUnit.SECONDS,          // keepAliveTime — how long an "extra" thread lives while idle
                new ArrayBlockingQueue<>(1),   // bounded queue: one slot
                new ThreadPoolExecutor.CallerRunsPolicy()); // rejection policy

        for (int i = 1; i <= 4; i++) {
            int number = i;
            pool.execute(() -> {
                System.out.println("task " + number + " — thread " + Thread.currentThread().getName());
                try {
                    Thread.sleep(300);
                } catch (InterruptedException e) {
                    Thread.currentThread().interrupt();
                }
            });
        }

        pool.shutdown();
        pool.awaitTermination(3, TimeUnit.SECONDS);
    }
}

The parameters work together:

1. While the number of threads is below corePoolSize, a new thread is created.
2. Once corePoolSize is reached, the task goes into the queue.
3. If the queue is full and there are fewer threads than maximumPoolSize, another thread is created.
4. If the queue is full and there are already maximumPoolSize threads, the rejection policy kicks in.

The printed order changes from run to run — the threads work in parallel — but the routing never does. Task 4 prints main: the pool refused it, and CallerRunsPolicy ran it on the submitting thread. A real service uses other numbers — four core threads, a queue of 200, eight at most — and the same routing.

Rejection policies

RejectedExecutionHandler decides what to do with a task that has nowhere to go:

• AbortPolicy (the default) — throws RejectedExecutionException.
• CallerRunsPolicy — the submitting thread runs the task itself.
• DiscardPolicy — the task is silently dropped.
• DiscardOldestPolicy — the oldest queued task is dropped and the new one takes its place.

For most services, CallerRunsPolicy is a sensible choice: instead of losing tasks, the system slows down intake on its own.

How many threads to create

There is no universal formula, but there are two poles:

CPU-bound tasks (computation, data processing without waiting): the pool size ≈ the number of CPU cores. Extra threads only create contention for the CPU.

int cores = Runtime.getRuntime().availableProcessors();
ExecutorService cpuPool = Executors.newFixedThreadPool(cores);

IO-bound tasks (database queries, HTTP calls, reading files): threads spend most of their time waiting for a response. The pool can be several times larger than the number of cores — while one thread waits on IO, others work. The usual "2×–4× the cores" is only a starting point.

Little's law gives a better number: tasks in flight equal the arrival rate multiplied by the average service time. A service making 600 HTTP calls per second, each waiting 0.4 s for a response, keeps 600 × 0.4 = 240 calls in flight — so the pool needs about 240 threads to keep its queue from growing. With eight threads ("two per core") the other 230 tasks sit in the queue, latency climbs into seconds, and the CPU stays idle — the threads wait on the network instead of computing. The core count sizes the pool only for CPU-bound work.

Shutting the pool down correctly

A pool is a resource; it must be closed. The JVM won't exit while the pool's non-daemon threads are alive.

pool.shutdown();

try {
    // wait no longer than 10 seconds
    if (!pool.awaitTermination(10, TimeUnit.SECONDS)) {
        pool.shutdownNow();
        // wait a bit more for the interruption
        pool.awaitTermination(5, TimeUnit.SECONDS);
    }
} catch (InterruptedException e) {
    pool.shutdownNow();
    Thread.currentThread().interrupt(); // restore the interrupt flag
}

The difference between the methods:

• shutdown() — the soft stop: new tasks are refused, queued and running ones finish.
• shutdownNow() — the hard stop: it calls interrupt() on every pool thread and returns the tasks that never ran. A task that ignores interruption keeps working.

In short

• Creating a thread for every task is wasteful — a pool reuses threads.
• ExecutorService is the main interface; execute for Runnable, submit for Callable/Future.
• The Executors factories suit prototypes; unbounded queues or thread counts are dangerous in production, so services use ThreadPoolExecutor with explicit parameters.
• A task's route: core thread → queue → a thread up to the maximum → rejection policy.
• CPU-bound: pool ≈ the number of cores; IO-bound: size the pool by Little's law, not by the core count.
• Always shut the pool down via shutdown() + awaitTermination() + shutdownNow() on timeout.

What to read next

• Threads and processes in Java — what a thread is and why it is expensive.
• CompletableFuture: asynchronous chains — how to build dependent asynchronous steps without manually waiting on a Future.
• Virtual Threads — an alternative to pools for IO-bound tasks in Java 21.
• Thread-safe collections — which data structures are safe to use between pool threads.