Functional tests answer "does it work". Load tests answer the questions that surface later and hurt more: how many users will the service survive, where does it break, and what happens at a spike. Answering them in production is expensive; that is what load testing is for.

load ramps up from left to right: 100 to 700 requests per second requests per second served latency 100 200 300 400 500 600 700 offeredsaturation pointthe service returns 450 per second target: p99 under 300 msp99 — 2.6 smean — 200 ms a 200 ms mean hides the tail: every hundredth request waits 2.6 seconds

While the load stays below the limit, exactly as many requests are served as were offered, and latency barely grows. After the saturation point the service still returns its 450 requests per second, while the extra ones wait in the queue — p99 bends upwards and reaches 2.6 s. The mean stays at 200 ms all the while: for every stuck request there are ninety-nine fast ones, and they pull it back down.

Kinds of load: four questions — four tests

  • Load test — "can we hold the expected traffic?": apply the planned load (say, 200 requests per second) and check whether latency stays within target.
  • Stress test — "where is the limit and how do we fail?": ramp the load until degradation. What matters is not only the ceiling but the character of the failure: graceful slowdown or an avalanche of errors and a crash.
  • Soak test — "will we survive the night?": moderate load for hours. It catches what ten minutes never show: memory leaks, connection exhaustion, swelling caches.
  • Spike test — "will we survive a burst?": a sharp jump (an ad aired, a campaign email went out) and a return to normal. It tests autoscaling and queues.

Metrics: what to measure

Throughput — requests per second (RPS/TPS). Errors — the share of non-2xx and timeouts. And above all — latency in percentiles:

  • p50 (median) — the typical user;
  • p95/p99 — the tail: every twentieth/hundredth request is slower than this.

The arithmetic mean is the worst latency metric: an average of 80 ms can hide a p99 of 4 seconds, and the suffering 1% on a large service is thousands of people. SLAs are stated in percentiles: "p99 < 300 ms at 500 RPS, errors < 0.1%".

Watch the pair together: as load grows, RPS climbs linearly, then flattens — and latency near the plateau shoots up. That is the system's limit (saturation).

Tools

JMeterGatlingk6
ScenariosGUI + XMLcode, Scala/Java DSLcode, JavaScript
Strengthmaturity, plugins, any protocolefficiency, built-in reportssimplicity, CI-friendliness, single binary
Weaknessheavy XML scenarios, costly threadsDSL learning curvefewer non-HTTP protocols
Fitsteams invested in it, exotic protocolsJVM teams, scenarios as codenew projects, load in the pipeline

Practical rule: for a new HTTP/gRPC project start with k6 or Gatling — scenarios live in git and run in CI. JMeter is chosen for exotic protocols and accumulated expertise.

Method: five steps

  1. A goal from the SLA: "p99 < 300 ms at 500 RPS" — without a goal the test cannot be interpreted.
  2. A realistic scenario: the production mix of operations (95% reads / 5% writes, not a uniform GET), realistic data, cache warm-up as a separate stage.
  3. A production-like stand: the same CPU/memory limits, the same database with production-sized data. A test against an empty database on an oversized stand measures nothing.
  4. The generator must not be the bottleneck: drive load from a separate machine (not a laptop over VPN) and verify it is the system that saturates, not the generator.
  5. Analysis at the plateau: application and infrastructure metrics (monitoring during the test) — CPU, DB connection pool, GC pauses: the limit always hits something specific.

Typical mistakes

  • Looking at the mean instead of percentiles — see above.
  • Measuring without warm-up: for the first minutes a JVM service is warming JIT and caches; those numbers say nothing about production.
  • Ignoring coordinated omission: if the generator waits for a response before sending the next request, it slows down together with the service and understates tail latency. Tools with an open workload model (k6 arrival rate, Gatling) protect against this.
  • A single run: a result without repetition is chance, not measurement.

In short

  • Four kinds: load — planned traffic, stress — finding the limit, soak — hours of work (leaks), spike — a sharp burst.
  • Metrics: RPS, error share, and latency percentiles (p50/p95/p99); the mean misleads, SLAs are written in percentiles.
  • Tools: k6 — JS scenarios and CI; Gatling — code and reports for JVM teams; JMeter — maturity and non-HTTP protocols.
  • Method: goal from SLA → realistic scenario → production-like stand → generator not the bottleneck → analysis at saturation.
  • The big traps: means instead of percentiles, no warm-up, coordinated omission, conclusions from one run.