A production service almost always has two requirements: it must handle growing load (traffic went up tenfold — and the site didn't go down) and not fail when things break (one server died — and customers never noticed). In AWS both problems are solved by one combination: several identical copies of the service sit behind a load balancer, spread across different data centers, and are automatically added and removed based on load.

That sounds like a lot of new terms — let's go through them one by one, with analogies and examples. After this article you'll understand what goes into a service that both handles spikes and survives failures.

Vertical and Horizontal Scaling

When load grows, you can expand in two ways.

Vertical scaling means taking a more powerful server: more CPU cores, more memory. Analogy: one cook in a café can't keep up — so you hire a more experienced cook who works twice as fast. Simple, but it has three downsides. First — every server has a ceiling: there's nothing more powerful than the most powerful option available. Second — to move to a bigger machine you usually have to restart the service (which means a pause in operation). Third, and most important — it's still just one server. If it breaks, the whole service goes down. This is called a single point of failure: one link whose failure brings down the entire system.

Horizontal scaling means adding more servers and distributing the load across them. Analogy: instead of one super-cook, you put five ordinary cooks in the kitchen and they split the orders. It's more complex to build, but it scales almost without limit (need more — add another couple), and one server breaking doesn't take down the service — the rest keep working.

The AWS way is horizontal. But it comes with a condition: the service must be stateless. That means the server doesn't keep important data in its own memory between requests. Why that matters: if there are five copies and a user's requests land randomly on one or another, none of them should "remember" something the others don't have. So state is moved outside — into a database or cache. Then the copies become interchangeable, and their number can be changed freely.

Auto Scaling Groups

An Auto Scaling Group (ASG) is the "copy manager" for your service. You give it three numbers:

  • minimum — how many copies to always keep running (for example, 2);
  • maximum — the ceiling you won't go above (for example, 10), so you don't go broke;
  • desired — how many are running right now.

From there the ASG adjusts the desired number based on load. The most common approach is target tracking: you say "keep average CPU usage around 50%," and the ASG adds copies when load climbs and removes them when it drops. The target doesn't have to be CPU — for example, ALBRequestCountPerTarget (the average number of requests per copy). There's also a smarter mode — predictive scaling: it looks at the history of the past couple of weeks and spins up copies ahead of time, before an expected peak (say, the morning rush), rather than after the fact.

The ASG's second, equally important talent is self-healing. It constantly checks the state of the copies. A copy stopped responding (failed its health check) — the ASG shuts it down and spins up a fresh one to replace it. That way the needed number of live copies is maintained without your involvement, even at night.

Load Balancers: ALB and NLB

There are now several copies — but the client knows one address. Someone has to greet each request and decide which copy to hand it to. That's the load balancer: a single front door hiding a pool of copies behind it. It distributes requests among them evenly and — an important detail — doesn't send traffic to unhealthy copies.

In AWS the load balancer family is called ELB (Elastic Load Balancing). For most tasks you pick one of two:

  • ALB (Application Load Balancer) works at layer 7 — the application layer (HTTP/HTTPS). It understands what's inside a request: it can route /api/* to some copies and /images/* to others, parse the domain name, decrypt HTTPS (terminate TLS). For websites and APIs this is the main choice.
  • NLB (Network Load Balancer) works at layer 4 — the transport layer (TCP/UDP). It doesn't look into the content, it just forwards connections very fast. It handles enormous loads with minimal latency and can have a static IP address. Needed for non-HTTP protocols and extreme traffic.

There are also more specialized types — the Gateway Load Balancer (for routing traffic through firewalls and security systems) and the legacy Classic Load Balancer (new projects aren't built on it). To get started it's enough to remember the ALB / NLB pair: HTTP — ALB, everything else and extreme loads — NLB.

By design the load balancer is placed in public subnets — the ones reachable from the internet — while the service copies themselves are hidden in private ones. From the outside only the load balancer is visible; the servers aren't exposed and can't be reached directly.

Multiple Zones and Health Checks

Horizontal scaling protects against one server failing. But what if an entire data center goes down — a fire, a power grid failure? If all your copies were sitting there, even the ASG won't help.

This is where Availability Zones (AZ) come into play. An AZ is a separate, physically isolated AWS data center. Within one region (say, Frankfurt) there are usually two or three such zones; they sit in different locations and don't depend on each other. If you spread the ASG's copies across several zones behind one load balancer, the failure of a whole zone stops being a catastrophe: the load balancer simply stops sending traffic there, and the copies in the live zones pick it up. This is what's called multi-AZ — deployment across several zones.

The mechanism that ties the whole construction together is health checks. Both the load balancer and the ASG regularly knock on each copy: "are you alive, ready to take requests?" It didn't respond as expected — the load balancer stops sending it traffic, and the ASG spins up a replacement. A subtlety beginners trip over: the check must be meaningful. A response of "the port is open" doesn't yet mean the service is ready to work — it may have started but not yet connected to the database. So you make a separate readiness endpoint that answers "ok" only when the service can actually serve requests. The exact same idea is used in Kubernetes with its liveness and readiness probes.

Where This Is Used

This combination — copies behind a load balancer, spread across zones, managed by Auto Scaling — is at the core of almost any production service in AWS: online stores, mobile app APIs, corporate portals. People build it by hand in the console when first getting acquainted, but in real projects it's described as code — via Terraform or CloudFormation, so the whole construction is reproducible and reviewable like ordinary code.

Typical mistakes made by beginners:

  • The service isn't stateless. Sessions and uploaded files are kept in a copy's memory — and on the second request to another copy the user gets "kicked out." Fixed by moving state into a database/cache and into object storage.
  • All copies in one zone. Multi-AZ wasn't enabled — the failure of a single data center takes down the whole service, even though there were several copies.
  • A meaningless health check. Checking "the port is open" instead of real readiness — the load balancer sends traffic to a copy that hasn't yet connected to the database, and users get errors.
  • The ASG maximum set sky-high. During a load spike (or a bug in the code, an infinite loop) copies multiply by the dozens — and a big bill arrives.

What to learn next. Start with the AWS fundamentals and networking if the terms "subnet" and "region" are still new. To automatically roll out new versions onto the copies without downtime, look at release strategies and build pipeline principles. Multi-AZ covers the failure of one zone, but not the failure of a whole region and not data loss — that's the topic of resilience and disaster recovery. And for bursty, unpredictable load it's sometimes better to skip your own servers entirely — look at serverless, where scaling happens on its own.