← Back to the section

Kubernetes and load balancers constantly ask your service: "are you alive? can I send requests?". If you don't answer correctly, the service either stops receiving traffic or, conversely, receives it when it isn't ready. Let's look at how to do this right in Go.

Why a single /health isn't enough

Imagine Postgres lags for 30 seconds because of a heavy operation. The service is alive, the process runs fine — the database is just temporarily unavailable.

If you have a single /health that returns DOWN when the database has problems, Kubernetes sees DOWN → restarts the pod → the new pod starts → the database is still lagging → the new pod is DOWN too → another restart. Within a minute all the pods are in a restart loop and the service is completely unavailable. And the problem was in the database, not in the process.

The solution — two separate endpoints with different semantics:

EndpointWhat it meansKubernetes reaction
/health/liveThe process is alive and not stuckDOWN → restart the pod
/health/readyThe service is ready to accept trafficDOWN → remove from load balancing

When the database lags: readiness → DOWN (traffic goes to other replicas), liveness stays UP (no restart). When the database recovers — readiness is UP again, traffic returns.

Liveness — just the process itself

Liveness checks one thing: the process is alive and not stuck in a deadlock. Nothing else. No databases, no external APIs.

The implementation is simple — it always returns 200:

func liveHandler(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusOK)
    _, _ = w.Write([]byte(`{"status":"UP"}`))
}

A common mistake is to add a database check to liveness. You must not do this: on any database lag Kubernetes will start restarting the pod, even though a restart won't improve the situation.

Readiness — readiness to accept traffic

Readiness checks that all external dependencies are available. If the database is unresponsive or the cache is unavailable, readiness returns 503 and Kubernetes temporarily removes the pod from the load balancer.

It's convenient to do this through an interface:

type Checker interface {
    Check(ctx context.Context) error
}

Then the readiness handler simply calls the checker:

type readyHandler struct {
    check Checker
}

func (h *readyHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    if err := h.check.Check(r.Context()); err != nil {
        w.WriteHeader(http.StatusServiceUnavailable)
        _ = json.NewEncoder(w).Encode(map[string]string{
            "status": "DOWN",
            "reason": "dependency unavailable",
        })
        return
    }
    w.WriteHeader(http.StatusOK)
    _, _ = w.Write([]byte(`{"status":"UP"}`))
}

A TTL cache for dependency checks

Kubernetes checks readiness every 5 seconds. If you have 10 replicas, that's 2 pings per second on Postgres from the health probe alone. Over time this becomes noticeable load.

The solution: cache the check result for a few seconds:

type pgChecker struct {
    db     *pgxpool.Pool
    mu     sync.Mutex
    lastOK time.Time
    ttl    time.Duration
}

func NewPostgresChecker(db *pgxpool.Pool, ttl time.Duration) Checker {
    return &pgChecker{db: db, ttl: ttl}
}

func (c *pgChecker) Check(ctx context.Context) error {
    c.mu.Lock()
    defer c.mu.Unlock()
    if time.Since(c.lastOK) < c.ttl {
        return nil // the last check was recent — don't ping again
    }
    if err := c.db.Ping(ctx); err != nil {
        return fmt.Errorf("postgres ping: %w", err)
    }
    c.lastOK = time.Now()
    return nil
}

A TTL of 10 seconds is a reasonable balance: on a real failure Kubernetes finds out within one or two checks, and the load on the database is minimal.

If there are several dependencies — a composite checker:

type compositeChecker struct {
    checkers []Checker
}

func NewComposite(checkers ...Checker) Checker {
    return &compositeChecker{checkers: checkers}
}

func (c *compositeChecker) Check(ctx context.Context) error {
    for _, ch := range c.checkers {
        if err := ch.Check(ctx); err != nil {
            return err
        }
    }
    return nil
}

Usage in main.go:

pgChecker := health.NewPostgresChecker(pgPool, 10*time.Second)
redisChecker := health.NewRedisChecker(redisClient, 10*time.Second)
ready := health.NewComposite(pgChecker, redisChecker)

/info — what's in production right now

During an incident the first question is "which version is running right now?". The /info endpoint answers this immediately:

{
  "version": "v1.4.2",
  "commit": "5380f21",
  "build_time": "2026-05-25T22:25:30Z"
}

The values are baked into the binary through compiler flags at build time:

BUILD_VERSION := $(shell git describe --tags --always)
BUILD_COMMIT  := $(shell git rev-parse --short HEAD)
BUILD_TIME    := $(shell date -u +%Y-%m-%dT%H:%M:%SZ)

build:
    go build \
      -ldflags="-X main.buildVersion=$(BUILD_VERSION) \
                -X main.buildCommit=$(BUILD_COMMIT) \
                -X main.buildTime=$(BUILD_TIME)" \
      ./cmd/order-service

In main.go we declare the variables with default values:

var (
    buildVersion = "dev"
    buildCommit  = "unknown"
    buildTime    = "unknown"
)

The handler for /info prepares the body once at startup:

func infoHandler(version, commit, buildTime string) http.HandlerFunc {
    body, _ := json.Marshal(map[string]string{
        "version":    version,
        "commit":     commit,
        "build_time": buildTime,
    })
    return func(w http.ResponseWriter, r *http.Request) {
        w.Header().Set("Content-Type", "application/json")
        _, _ = w.Write(body)
    }
}

The management server on a separate port

Keep the health endpoints on a separate port (usually 9090), not on the business port (8080). This lets you close off /metrics and /health from external clients with a network policy — Kubernetes sees them, the internet doesn't.

The function that starts the management server:

func StartManagement(addr string, ready Checker) *http.Server {
    mux := http.NewServeMux()
    mux.Handle("/metrics", promhttp.Handler())
    mux.HandleFunc("/health/live", liveHandler)
    mux.Handle("/health/ready", &readyHandler{check: ready})
    mux.HandleFunc("/info", infoHandler(buildVersion, buildCommit, buildTime))
    return &http.Server{Addr: addr, Handler: mux}
}

The business server and the management server start in parallel via errgroup:

g, gCtx := errgroup.WithContext(ctx)
g.Go(func() error {
    if err := businessSrv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
        return fmt.Errorf("business server: %w", err)
    }
    return nil
})
g.Go(func() error {
    if err := managementSrv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
        return fmt.Errorf("management server: %w", err)
    }
    return nil
})
g.Go(func() error {
    <-gCtx.Done()
    _ = businessSrv.Shutdown(context.Background())
    _ = managementSrv.Shutdown(context.Background())
    return nil
})
return g.Wait()

When the process terminates (SIGTERM): readiness switches to DOWN — new requests stop arriving, already-started requests finish processing, and the process exits cleanly.

Kubernetes manifest for the probes:

spec:
  containers:
    - name: order-service
      livenessProbe:
        httpGet:
          path: /health/live
          port: 9090
        initialDelaySeconds: 10
        periodSeconds: 10
        failureThreshold: 3
      readinessProbe:
        httpGet:
          path: /health/ready
          port: 9090
        initialDelaySeconds: 5
        periodSeconds: 5
        failureThreshold: 2

Common mistakes

Business state in readiness. pendingOrders > 1000 → DOWN is wrong. Health reports the technical state of the process: the database is available, connections are established. Business metrics (queues, delays) are SLOs and Prometheus alerts, not health.

Liveness checks the database or Redis. On any external-system failure Kubernetes will start restarting the pod. A restart won't help — the database is the same. Liveness must depend only on the process itself.

A probe triggers a business operation. INSERT INTO product_checks in a health probe is a bad idea: 20 replicas × every 5 seconds = constant load on the database. The right way: db.Ping() or a TTL-cached result.

A checker without a TTL cache. Without a cache the probe makes a real database query on every check. With a TTL of 10 seconds the load drops by a factor of 10.

In short

  • Two endpoints with different semantics: /health/live — the process is alive, /health/ready — ready for traffic.
  • Liveness depends only on the process itself — no databases and no external APIs.
  • Readiness checks external dependencies and returns 503 if they're unavailable.
  • A TTL cache in the checker reduces the load that the health probe puts on dependencies.
  • A composite checker combines several dependencies into a single readiness check.
  • /info with version, commit, build_time — first aid during an incident.
  • The management server on a separate port: Kubernetes sees it, external clients don't.
  • Logging in Go — slog, JSON and context.Context.
  • Metrics in Go — promauto, RED middleware on chi, business metrics.
  • Tracing in Go — otelhttp, otelpgx, sampling.