Every time you restart a service in Kubernetes — deploying a new version, scaling down, restarting a pod — some requests are "in flight" at the exact moment of shutdown. Without special configuration uvicorn cuts them off, and the client gets a 502 or connection reset. This is not a framework bug, it's the default behaviour that needs to be fixed.
What happens on shutdown without configuration
Kubernetes decides to stop the pod. It sends the process a SIGTERM signal. uvicorn receives the signal and immediately terminates all active connections. Clients whose requests were being processed get a connection break.
In parallel, Kubernetes updates the routing — it removes the pod from the list of live endpoints. But this doesn't happen instantly: kube-proxy on other nodes updates its iptables rules over 5–15 seconds. During this window new requests can still arrive at the already-dying pod.
The result: up to 1–2% of requests are lost on every deploy if you don't take measures.
How uvicorn waits for in-flight requests
uvicorn can shut down correctly — you just need to enable the timeout_graceful_shutdown parameter. On receiving SIGTERM it stops accepting new connections, but active handlers continue their work until completion (or until the timeout expires).
# main.py
import uvicorn
if __name__ == "__main__":
uvicorn.run(
"app:app",
host="0.0.0.0",
port=8080,
timeout_graceful_shutdown=30,
)
Or via the command line when starting the container:
uvicorn app:app --host 0.0.0.0 --port 8080 --timeout-graceful-shutdown 30
A value of 0 or a missing parameter means uvicorn breaks connections instantly. That's not a graceful shutdown.
Readiness probe — the first "I'm dying" signal
To make Kubernetes stop sending traffic to a dying pod faster, the readiness probe should start returning 503 at the very beginning of the shutdown — before connections are actually closed.
For this we use the FastAPI lifespan: the ready flag is set to False first thing on shutdown.
# app/state.py
from dataclasses import dataclass
@dataclass
class AppState:
ready: bool = False
# app/lifespan.py
from contextlib import asynccontextmanager
from fastapi import FastAPI
from app.state import AppState
app_state = AppState()
@asynccontextmanager
async def lifespan(app: FastAPI):
# start: initialization
await kafka_consumer.start()
app_state.ready = True
yield
# shutdown: flag first, then close resources
app_state.ready = False
await engine.dispose()
await kafka_consumer.stop()
# app/health.py
from fastapi import APIRouter, Response
from app.lifespan import app_state
router = APIRouter()
@router.get("/health/ready")
async def readiness():
if not app_state.ready:
return Response(status_code=503)
return {"status": "ok"}
@router.get("/health/live")
async def liveness():
return {"status": "ok"}
As soon as the readiness probe sees a 503, Kubernetes stops sending new requests to this pod. Active handlers meanwhile continue their work — uvicorn doesn't cut them off.
preStop sleep — why wait before SIGTERM
Even with a correct uvicorn graceful shutdown there is a window of 5–15 seconds during which kube-proxy on other nodes hasn't yet updated the routing rules. In those seconds new traffic keeps arriving at a pod that has already begun shutting down and isn't accepting connections — the client gets a 502.
The solution is simple: delay sending SIGTERM by 10 seconds via a preStop hook. During that time kube-proxy manages to update, and by the moment of actual shutdown new traffic is no longer arriving.
spec:
containers:
- name: order-service
lifecycle:
preStop:
exec:
command: ["sh", "-c", "sleep 10"]
terminationGracePeriodSeconds: 60
What happens with this configuration:
T=0 Kubernetes decides to stop the pod
T=0+ kubelet runs the preStop hook (sleep 10)
Simultaneously: the endpoints controller removes the pod from the Service
Simultaneously: kube-proxy updates iptables on all nodes
T=10s preStop finished — all rules are already updated
T=10s kubelet sends SIGTERM
T=10s+ uvicorn graceful: finishes in-flight requests, accepts only those
During these 10 seconds the application keeps processing requests normally — SIGTERM hasn't arrived yet. This is important: sleep doesn't stop the application, it only delays the shutdown signal.
terminationGracePeriodSeconds: 60 is Kubernetes's overall budget for the whole process. It must be larger than preStop sleep + timeout_graceful_shutdown (10 + 30 = 40 seconds), otherwise Kubernetes will forcibly kill the pod earlier.
On large clusters (1000+ nodes) kube-proxy can take up to 20 seconds to update — then the sleep should be increased to 20.
Long endpoints — a special problem
Imagine an endpoint that runs for 30 seconds: generating a large report, a complex calculation. On SIGTERM uvicorn waits for active handlers to finish for at most timeout_graceful_shutdown seconds. If three such requests arrived at the same time and all run for 30 seconds, half won't finish within the budget and will be interrupted.
Option 1: 202 Accepted and polling
The cleanest way. Instead of keeping the HTTP connection open for 30 seconds, the endpoint accepts the task and immediately returns 202 with an identifier. The client periodically asks about the status.
# app/orders/router.py
import uuid
from fastapi import APIRouter, BackgroundTasks, Depends, Header
router = APIRouter(prefix="/orders")
@router.post("/reports", status_code=202)
async def request_report(
background_tasks: BackgroundTasks,
idempotency_key: str = Header(..., alias="Idempotency-Key"),
service: OrderReportService = Depends(get_service),
) -> dict:
report_id = str(uuid.uuid4())
background_tasks.add_task(
service.generate_report,
report_id=report_id,
idempotency_key=idempotency_key,
)
return {"report_id": report_id, "status": "QUEUED"}
@router.get("/reports/{report_id}")
async def get_report(
report_id: str,
service: OrderReportService = Depends(get_service),
) -> dict:
return await service.get_report_status(report_id)
The POST returns a response in less than a second. The client periodically calls GET until it receives status: READY. On service shutdown the current iteration of the background task is finished and no new requests arrive.
The Idempotency-Key header lets the client safely retry the request when in doubt — no duplicates will arise.
Option 2: explicit asyncio.Task with awaiting on shutdown
For services with intensive background work — explicit task control through asyncio.Task and awaiting their completion in lifespan:
# app/lifespan.py
import asyncio
from contextlib import asynccontextmanager
from fastapi import FastAPI
from app.state import app_state
running_tasks: set[asyncio.Task] = set()
@asynccontextmanager
async def lifespan(app: FastAPI):
app_state.ready = True
yield
app_state.ready = False
if running_tasks:
await asyncio.wait(running_tasks, timeout=25.0)
await engine.dispose()
await kafka_consumer.stop()
def create_tracked_task(coro) -> asyncio.Task:
task = asyncio.create_task(coro)
running_tasks.add(task)
task.add_done_callback(running_tasks.discard)
return task
# usage in the router
@router.post("/products/{product_id}/sync", status_code=202)
async def sync_product(product_id: str) -> dict:
create_tracked_task(product_sync_service.sync(product_id))
return {"status": "ACCEPTED"}
When asyncio cancels a task on timeout, it gets a CancelledError. Critical sections need to be handled explicitly: finish the transaction, then re-raise the exception.
async def sync_product_inventory(product_id: str) -> None:
async with db_session() as session:
try:
await session.execute(update_product_query(product_id))
await session.commit()
except asyncio.CancelledError:
await session.rollback()
raise
Common mistakes
--timeout-graceful-shutdown 0 or not set — uvicorn breaks all active connections instantly on SIGTERM. The client sees a connection reset. Always set 30 or higher.
No preStop sleep — even with a correct uvicorn graceful shutdown, new traffic arrives at the dying pod within the 5–15 second window. Guaranteed 502s. At minimum sleep 10.
sleep shorter than 5 seconds on a large cluster — kube-proxy doesn't manage to update on all nodes. On clusters with 1000+ nodes you need 20 seconds.
A synchronous endpoint without conversion to async — a long endpoint blocks the drain and still gets cut off on timeout. The 202 Accepted pattern solves this.
httpGet in preStop instead of exec sleep — HTTP responses are unreliable during pod shutdown. Only exec: ["sh", "-c", "sleep N"].
In short
- uvicorn waits for in-flight requests with
--timeout-graceful-shutdown 30. Without this parameter — instant break. - The readiness probe switches to 503 first thing in the
lifespanshutdown — via theapp_state.ready = Falseflag. preStop: exec: sleep 10in Kubernetes is mandatory: kube-proxy updates its rules 5–15 seconds after SIGTERM, and in that window, without the sleep, new traffic arrives at the dying pod.terminationGracePeriodSecondsmust be larger than the sum ofpreStop sleep+timeout_graceful_shutdown.- A long synchronous endpoint > 10 seconds — use the 202 Accepted + polling pattern, or explicit
asyncio.Taskwith awaiting in lifespan. CancelledErrorin tasks — handle it explicitly: finish the transaction, thenraise.
What to read next
- uvicorn and lifespan configuration — full startup parameters, separate health probes.
- Kubernetes and terminationGracePeriodSeconds — details of configuring the pod for graceful shutdown.
- Idempotency of in-flight requests — how a client safely retries a request when a service restarts.
- Database and persistence — the order of calling
engine.dispose()on shutdown.