← Back to the section

Once the container image is built and pushed to a registry, the interesting part is just beginning. You have to explain to Kubernetes how many copies to run, when to consider a pod ready, how to update without downtime, and where to look if something goes wrong. All of this is described in files — and the files live in git.

Manifest: one file — one intention

Services used to be laid out on servers by hand or with scripts. Kubernetes works differently: you describe what you want, and the cluster figures out how to make it happen.

The description is written in YAML files called manifests. A typical pair for one service is two files:

# deployment.yaml — how many to run and which image
apiVersion: apps/v1
kind: Deployment
metadata:
  name: order-service
spec:
  replicas: 3
  selector:
    matchLabels:
      app: order-service
  template:
    metadata:
      labels:
        app: order-service
    spec:
      containers:
        - name: app
          image: order-service:1.42.0
          ports:
            - containerPort: 8080

# service.yaml — how to reach the pods from inside the cluster
apiVersion: v1
kind: Service
metadata:
  name: order-service
spec:
  selector:
    app: order-service
  ports:
    - port: 80
      targetPort: 8080

Applying a manifest is a single command:

kubectl apply -f deployment.yaml

Kubernetes reads the file and brings the cluster to the described state. If a pod dies — the cluster brings up a new one. If you change the number of replicas and apply the file again — the cluster rebalances on its own. This is what's called the declarative approach.

Kustomize and Helm: when one file isn't enough

A single file works well as long as you have one environment. As soon as staging and prod appear, they differ in the number of replicas, resource limits, database addresses. Copying the manifest and changing a few lines is a path to drift.

There are two tools that solve this problem in different ways.

Kustomize works without templates: there's a base variant and "overlays" for each environment. You can overlay anything: just replicas, just resources, just the image address. Kustomize is built right into kubectl:

deploy/
  base/                 # deployment.yaml, service.yaml — common foundation
  overlays/
    staging/            # replicas: 1, less memory
    prod/               # replicas: 3, HPA, PodDisruptionBudget

kubectl apply -k overlays/prod/

Helm is already a templating engine with packaging. From a single chart you can deploy a service dozens of times with different parameters. This is exactly how all third-party software (databases, monitoring, ingress controllers) works — you download a chart and pass a values.yaml for your environment.

helm upgrade --install order-service ./chart -f values-prod.yaml

The downside of Helm is that templates are written in Go templates on top of YAML. This is harder to read, and for debugging you often need to see what Helm generated:

helm template order-service ./chart -f values-prod.yaml

The practical choice: kustomize is convenient for your own services, where there aren't many changes; Helm is needed for third-party software and when the configuration really varies a lot between environments.

Image tags: how not to lose the ability to roll back

A small detail with big consequences. The :latest tag on an image looks convenient — push a new image under the same tag, and that's it. But it breaks three important things:

  • rollback: what do you roll back to if the tag is the same?
  • reproducibility: what exactly is in prod right now?
  • rolling update: kubelet may decide that the image "hasn't changed" and won't restart the pod.

There's one rule: a tag is unique and immutable. Good options are a version (order-service:1.42.0) or a commit SHA (order-service:a1b2c3d). Then at any moment you can see what's deployed, and you can return to any previous state.

Rolling update: how a version reaches prod without downtime

By default Kubernetes updates pods gradually, without shutting down the entire service. This is called a rolling update: new pods come up, old ones are shut down in batches.

The behavior is configurable:

strategy:
  rollingUpdate:
    maxSurge: 1        # how many extra pods can be created above replicas
    maxUnavailable: 0  # how many pods may be down during the process

maxUnavailable: 0 means: capacity never dips — a new pod starts before an old one stops. This is the conservative option for prod.

Here the key dependency appears: a new pod receives traffic only after it passes the readiness probe. This is a special check that Kubernetes performs before letting requests reach the pod. Until the pod answers "ready" — it receives no traffic.

If the probe is honest (it checks the connection to the database, the queue, the required dependencies) — a broken version simply won't pass the check, and the rollout will stop. kubectl rollout status will show the problem.

If the probe is a formality ("I always answer 200") — a broken version will immediately accept traffic, and you have an incident.

Rolling back when there's a problem is a routine operation:

kubectl rollout undo deployment/order-service
kubectl rollout history deployment/order-service   # revision history

Besides rolling update, there's the Recreate strategy — first shut down all old pods, then bring up new ones. This causes brief downtime, but is needed for background workers that must not run in two versions at the same time.

Database migrations: when two versions run in prod at once

During a rolling update, the old and new versions of the application coexist in the cluster for a few minutes. This means both versions work with the same database at the same time.

Hence the rule: every migration must be compatible with the old version of the application. Renaming a column with RENAME COLUMN is not allowed — the old version won't find it. Instead, use the expand-contract approach across several releases:

  1. Release 1: add the new column, the old one stays.
  2. Release 2: both codebases write to both columns, read from the new one.
  3. Release 3: the old column is removed.

Migrations are usually run through an init container — a special container in the pod that starts before the main application and finishes before the application opens its port. Kubernetes waits for the init container to complete successfully before starting the main one.

GitOps: git as the source of truth about the cluster state

A regular CI/CD pipeline works like this: tests → build image → push to registry → kubectl apply from the pipeline. This is called the push approach: CI itself pushes changes into the cluster. The downside — the cluster is accessible from CI, and the actual state isn't recorded anywhere.

GitOps flips this: git stores the "desired state" of the cluster, and a special operator in the cluster (Argo CD, Flux) continuously reconciles the actual state with git and applies the difference itself.

Deploying a new version in GitOps is a pull request with a new image tag. Rollback is a git revert. If someone manually changes something in the cluster — the operator notices it as a "deviation from desired" (drift) and reverts it back.

developer → git push (new tag) → Argo CD sees the difference → applies to the cluster

GitOps works well when there are more than two or three services: a change history in git, an audit trail, rollbacks through standard git tools.

In short

  • A manifest is a YAML file with a declaration: "I want 3 copies of this image." Kubernetes ensures this state on its own.
  • Kustomize is overlays without templates, for your own services with several environments. Helm is a templating engine with packaging, needed for third-party software.
  • An image tag should be unique and immutable — a version or a commit SHA. The :latest tag breaks rollbacks and reproducibility.
  • A rolling update updates pods in batches without stopping the service. Traffic reaches a new pod only after a green readiness probe.
  • kubectl rollout undo rolls back to the previous revision — a routine operation, not a feat.
  • During a rolling update, two versions run in the cluster at the same time, so database migrations must be compatible with the old code.
  • GitOps: git stores the desired state of the cluster, an operator (Argo CD, Flux) applies it automatically. Deployment = a PR with a new tag.
  • Spring Boot in Kubernetes — readiness probe via Actuator, graceful shutdown, JVM resources.
  • Operations and debugging — what to do when a rollout is stuck.
  • Kubernetes fundamentals — pods, nodes, controllers, and desired state.