Graceful Shutdown

You roll out a new version of a service. Kubernetes stops the old pod — and some requests abort with an error, a transaction is left half-done, a Kafka message is lost. Every deploy turns into a lottery: lucky or not.

Graceful shutdown is a set of techniques that let a service finish what it started as it stops, losing not a single request. When the "terminate" signal (SIGTERM) arrives, the service does not crash instantly: it stops accepting new traffic, finishes the requests already in flight, carefully stops background tasks and consumers, closes database connections — and only then exits.

This section breaks graceful shutdown down by the layers it depends on:

• Application configuration — what to enable so the server actually starts terminating cleanly instead of tearing connections.
• HTTP draining — how to finish active requests and why traffic is still lost in Kubernetes without a preStop.
• Database and transactions — what happens to the connection pool and unfinished transactions.
• Background tasks, @Async and the outbox — how to stop periodic tasks without cutting them off midway.
• Kafka and consumers — how to finish reading the current batch of messages and not lose the offset.
• Idempotency and retries — why, without idempotency, shutting down can lead to double operations.
• Kubernetes — probes, preStop, terminationGracePeriodSeconds, and the order of events during a rollout.
• Budgets and observability — how much time to budget for termination and how to see that it went cleanly.

The articles come in variants for different languages and stacks — pick your own.