Thread Pools: ExecutorService

← Back to the section

Creating a new thread for every task is expensive and dangerous. ExecutorService solves this: it keeps ready threads on hand and distributes work among them.

The problem: why you need a pool at all

A thread in Java is an expensive object. Creating one takes milliseconds and requires allocating memory for the stack (512 KB–1 MB by default). If an application creates a thread for every HTTP request or background task, under load this leads to two problems:

• Memory overuse. A thousand concurrent threads means gigabytes spent on stacks alone.
• Scheduler overload. The operating system spends more time switching between threads than doing useful work.

A thread pool is a set of pre-created threads that wait for tasks in a queue. A task arrives → a free thread picks it up → runs it → returns to wait for the next one. A thread is created once, not on every request.

ExecutorService: the base interface

ExecutorService is the main interface for managing a pool. It extends Executor, adding the ability to submit tasks that return a result and to manage the lifecycle.

Two main ways to hand off a task:

ExecutorService pool = Executors.newFixedThreadPool(4);

// execute — for tasks without a result (Runnable)
pool.execute(() -> System.out.println("background work"));

// submit — for tasks with a result (Callable) or Runnable
Future<Integer> future = pool.submit(() -> {
    // the computation takes time
    return 42;
});

Integer result = future.get(); // blocks until the result is ready

Callable<V> differs from Runnable in that it can return a value and throw a checked exception. Future<V> is a "receipt": the task is still running, but the result can be retrieved later via future.get().

The Executors factories and their limits

The Executors class provides ready-made factories:

Why an explicit ThreadPoolExecutor is better in production. newFixedThreadPool has an unbounded queue (LinkedBlockingQueue with no limit). If tasks arrive faster than they are processed, the queue grows until memory is exhausted. newCachedThreadPool has no upper bound on the number of threads: during a load spike the system can create thousands of threads.

ThreadPoolExecutor: full control

ThreadPoolExecutor pool = new ThreadPoolExecutor(
    4,                              // corePoolSize — permanent threads
    8,                              // maximumPoolSize — the cap under peak load
    30, TimeUnit.SECONDS,           // keepAliveTime — how long "extra" threads live
    new ArrayBlockingQueue<>(200),  // bounded queue
    new ThreadPoolExecutor.CallerRunsPolicy() // rejection policy
);

The parameters work together:

1. While the number of threads is below corePoolSize, a new thread is created.
2. Once corePoolSize is reached, the task is placed in the queue.
3. If the queue is full and there are fewer threads than maximumPoolSize, another thread is created.
4. If the queue is full and there are already maximumPoolSize threads, the rejection policy kicks in.

Rejection policies

RejectedExecutionHandler decides what to do with a task that has nowhere to go:

• AbortPolicy (the default) — throws RejectedExecutionException.
• CallerRunsPolicy — the task is run by the very thread that submitted it (a natural slowdown of the submitter).
• DiscardPolicy — the task is silently dropped.
• DiscardOldestPolicy — the oldest task is removed from the queue and the new one takes its place.

For most services, CallerRunsPolicy is a sensible choice: instead of losing tasks, the system slows down intake on its own.

How many threads to create

There is no universal formula, but there are two poles:

CPU-bound tasks (computation, data processing without waiting): the pool size ≈ the number of CPU cores. Extra threads only create contention for the CPU.

int cores = Runtime.getRuntime().availableProcessors();
ExecutorService cpuPool = Executors.newFixedThreadPool(cores);

IO-bound tasks (database queries, HTTP calls, reading files): threads spend most of their time waiting for a response. The pool can be several times larger than the number of cores — while one thread waits on IO, others work. Typical rules of thumb: 2×–4× the cores for moderate IO, but the exact value is found through load testing.

// for IO-bound: more threads than cores
ExecutorService ioPool = new ThreadPoolExecutor(
    cores * 2, cores * 4,
    60, TimeUnit.SECONDS,
    new ArrayBlockingQueue<>(500),
    new ThreadPoolExecutor.CallerRunsPolicy()
);

Shutting the pool down correctly

A pool is a resource; it needs to be closed. If you don't stop the pool, the JVM won't exit while the pool's threads are alive (unless they are daemon threads).

pool.shutdown(); // stops accepting new tasks, waits for current ones to finish

try {
    // wait no longer than 10 seconds
    if (!pool.awaitTermination(10, TimeUnit.SECONDS)) {
        pool.shutdownNow(); // interrupt the unfinished tasks
        // wait a bit more for the interruption
        pool.awaitTermination(5, TimeUnit.SECONDS);
    }
} catch (InterruptedException e) {
    pool.shutdownNow();
    Thread.currentThread().interrupt(); // restore the interrupt flag
}

The difference between the methods:

• shutdown() — a "soft" stop: no new tasks are accepted, but the ones already queued and running are finished.
• shutdownNow() — a "hard" stop: it calls interrupt() on all pool threads and returns the list of tasks that didn't get to run. A task must respond to interruption, otherwise the thread keeps working.

Diagram of a task's life in the pool

In short

• Creating a thread for every task is wasteful — a pool reuses threads.
• ExecutorService is the main interface; execute for Runnable, submit for Callable/Future.
• The Executors factories are handy for prototypes, but their queues or thread counts are unbounded — dangerous in production.
• ThreadPoolExecutor with explicit corePoolSize, maximumPoolSize, a bounded queue, and a rejection policy is the right choice for services.
• CPU-bound: pool ≈ the number of cores; IO-bound: pool 2–4 times larger, with the exact value coming from load testing.
• Always shut the pool down via shutdown() + awaitTermination() + shutdownNow() on timeout.

What to read next

• CompletableFuture: asynchronous chains — how to build dependent asynchronous steps without manually waiting on a Future.
• Virtual Threads — an alternative to pools for IO-bound tasks in Java 21.
• Thread-safe collections — which data structures are safe to use between pool threads.