Kubernetes and graceful shutdown: terminationGracePeriodSeconds, probes, maxSurge

← Back to the section

You configured Spring graceful shutdown — the application drains requests correctly when it receives SIGTERM. You deploy. Clients still see 502s. Why?

Because Spring and Kubernetes terminate the pod independently of each other, and without the right configuration K8s keeps routing requests to a pod that no longer responds. Let's break down exactly what to configure and why.

What happens when a pod is terminated

When Kubernetes decides to terminate a pod (during a new-version deploy, a scale-down, or an explicit delete), the following happens:

1. The kubelet runs the preStop hook (if present).
2. At the same time, K8s starts removing the pod from the endpoints list — that is, it takes it out of load balancing.
3. After preStop, the kubelet sends SIGTERM to the process.
4. The process performs a graceful shutdown and exits.
5. If the process hasn't exited within terminationGracePeriodSeconds, the kubelet sends SIGKILL.

The problem is in step 2: kube-proxy on other nodes doesn't update its routing tables instantly. Between the moment K8s marks the pod as terminating and the moment new requests stop arriving, 5–15 seconds pass. All of those requests will get a 502.

preStop: a buffer against 502s

preStop is a hook the kubelet runs before sending SIGTERM. If you put a sleep 10 in it, the application gets 10 seconds while kube-proxy propagates the change across the cluster, and only then does termination begin.

spec:
  containers:
    - name: app
      image: order-service:1.4.2
      lifecycle:
        preStop:
          exec:
            command: ["sh", "-c", "sleep 10"]

Without preStop, SIGTERM is sent immediately. Spring begins shutdown and stops accepting new requests. But kube-proxy doesn't know yet — and for another 5–15 seconds it keeps sending requests that get rejected.

On a single pod that's several thousand errors. On a rolling deploy with a dozen pods, it happens on every one of them.

terminationGracePeriodSeconds: why 60, not 30

terminationGracePeriodSeconds is the total time budget from the start of preStop to a forced SIGKILL. The K8s default is 30 seconds.

Let's do the math:

• preStop sleep 10 — 10 seconds.
• Spring graceful shutdown — needs at least 30 seconds (the Spring default).
• Total: 10 + 30 = 40 seconds.

40 seconds don't fit into a 30-second budget. The pod gets SIGKILL in the middle of the drain — active requests are aborted.

The correct configuration:

spec:
  terminationGracePeriodSeconds: 60
  containers:
    - name: app
      image: order-service:1.4.2
      lifecycle:
        preStop:
          exec:
            command: ["sh", "-c", "sleep 10"]

60 seconds give a comfortable margin: 10 for kube-proxy, 30 for draining requests, and another 20 for unforeseen delays.

The full sequence of events:

T=0      Kubelet runs preStop
T=0      K8s starts removing the pod from endpoints
T=10s    preStop done → kubelet sends SIGTERM
T=10..40s  Spring drain: finishes in-flight requests
T=40s    Process exits
         (If not — SIGKILL at T=60s)

readiness probe and liveness probe: different goals

The two probe types do different things on failure:

During shutdown, readiness is what matters:

1. Spring publishes the state "refusing traffic".
2. The /actuator/health/readiness endpoint starts responding with 503.
3. A few seconds later K8s sees the readiness failure and removes the pod from endpoints.
4. Traffic stops going to this pod.

If the liveness probe is pointed at the same endpoint and also fails during shutdown, K8s restarts the pod instead of letting it terminate cleanly. That breaks the entire graceful shutdown.

The correct configuration:

readinessProbe:
  httpGet:
    path: /actuator/health/readiness
    port: 8080
  periodSeconds: 5
  failureThreshold: 2
livenessProbe:
  httpGet:
    path: /actuator/health/liveness
    port: 8080
  periodSeconds: 10
  failureThreshold: 3
  initialDelaySeconds: 60

initialDelaySeconds: 60 on liveness protects against a restart during startup. A JVM with warm-up and its first requests takes 20–30 seconds to start; without the delay, the liveness probe can kill a pod that is simply still loading.

Readiness without a delay is fine: while the pod isn't ready, a 503 just means "don't send traffic here".

maxSurge and maxUnavailable: zero-downtime deploy

A rolling deploy replaces old pods with new ones gradually. Two parameters control exactly how:

strategy:
  type: RollingUpdate
  rollingUpdate:
    maxSurge: 1
    maxUnavailable: 0

• maxSurge: 1 — K8s may create one pod beyond the specified number of replicas. With 3 replicas, a 4th pod appears during the deploy: the new one starts up before the old one begins terminating.
• maxUnavailable: 0 — no pod may be unavailable at any one time.

What happens during a deploy:

Start: 3 pods (version v1)

1. A v2 pod is created, 4 pods total
2. The v2 pod passes the readiness probe → enters endpoints
3. One v1 pod begins terminating:
   preStop sleep → SIGTERM → drain → exit
   Total: 2 v1 + 1 v2 = 3 active
4. A second v2 pod is created, 4 pods total
...

A new pod always enters the rotation before an old one begins terminating. Capacity never drops below the requested number of replicas.

If you set maxUnavailable: 1, K8s may kill an old pod before the new one starts. At that moment there will be 2 active pods instead of 3, which under peak load can lead to 503s.

Common mistakes

No preStop. Without the hook, SIGTERM is sent immediately, kube-proxy hasn't updated yet — 5–15 seconds of guaranteed 502s on every pod being restarted.

terminationGracePeriodSeconds: 30 with preStop sleep 10. Spring is left with only 20 seconds instead of 30. Requests that run longer are aborted by SIGKILL.

A single /actuator/health endpoint for both probes. During shutdown, a 503 on liveness triggers a pod restart — graceful shutdown never completes and a restart loop begins.

A liveness probe that depends on the database. If the DB is unavailable, liveness fails and the pod is restarted. The right way: liveness checks only that the process is alive, not the availability of dependencies.

initialDelaySeconds: 0 on liveness. The pod is killed right after startup, while the JVM is still warming up. Use at least 30 seconds, 60 is safer.

In short

• preStop sleep 10 — a mandatory buffer: kube-proxy updates routes for 5–15 seconds, and without it requests go to a terminating pod and get 502s.
• terminationGracePeriodSeconds: 60 — the default 30 isn't enough: preStop 10 + Spring graceful 30 = 40 seconds, and the pod is killed in the middle of the drain.
• readiness probe removes the pod from endpoints without a restart. liveness probe restarts the pod. On graceful shutdown you need readiness=503, and liveness must not fail at that point.
• maxSurge: 1, maxUnavailable: 0 — a new pod enters the rotation before an old one begins terminating. Capacity doesn't sag.
• initialDelaySeconds: 60 on liveness protects against a restart during JVM warm-up.

Further reading

• HTTP drain and preStop — how Spring drains requests in the window between SIGTERM and exit.
• JVM and Spring configuration — how to set the Spring graceful timeout and what it means.
• Budgets and observability — how to split 60 seconds across the phases.