Managed Data in AWS: RDS, ElastiCache, SQS, MSK

A "managed" service is when you use a technology while the provider takes care of running it. Your own database on a server means you install PostgreSQL yourself, configure backups, and fix things at night when the disk fills up. A managed database is that same PostgreSQL, but backups, replacing a failed node, and version upgrades are handled by AWS. You pay for it with money, but not with time and not with sleepless nights.

The main value of the cloud for a backend developer is not virtual machines, but managed data specifically: databases, caches, queues, and message brokers where the routine is offloaded to the provider. This article is a map of correspondences — "what you know from your own server → what it's called in AWS" — and an honest list of what a managed service does not do for you.

RDS and Aurora: PostgreSQL as a service

RDS (Relational Database Service) is relational databases managed by AWS. RDS for PostgreSQL is the very same PostgreSQL you're used to, only with automation wrapped around it:

• Backups with the ability to restore to any second in the past (Point-In-Time Recovery, PITR).
• A replica in another availability zone (Multi-AZ): if the primary server dies, AWS automatically switches the load to the standby. An availability zone (AZ) is a separate data center within one region; there are usually several of them, so a fire in one building doesn't bring down the service.
• Version upgrades during a maintenance window you set in advance.

Aurora is a database built by AWS, compatible with PostgreSQL at the protocol level but rewritten under the hood. Its key trick is a separate distributed storage layer: each piece of data is stored in six copies across three availability zones (two copies per zone). A write is acknowledged once four of the six copies have accepted it, so Aurora survives the loss of two copies without pausing writes. Replicas read from the same shared storage as the primary, so replica lag is nearly zero, and failover usually takes less than a minute. You pay for this with a price premium and a small loss of the "it's just PostgreSQL" feeling — separate parameters, its own behavior under load.

Here's what a beginner needs to understand: a managed database takes hardware operations off your shoulders, but not data design. RDS will save you from a burned-out disk, but not from a SELECT query without an index that scans a million rows. Table schema, indexes, locks, safe structural changes without downtime, tuning slow queries — all of that is still your job.

The rake almost everyone steps on: the connection limit. A database has a maximum number of simultaneous connections, and it depends on the server's memory size. When a dozen applications each hold a pool of 10 connections, the limit gets eaten up unnoticed, and new connections start failing with an error. There are two cures: discipline around pool sizes and a component called RDS Proxy — a layer that multiplexes (reuses) connections across applications. RDS Proxy is practically mandatory if there are functions running nearby that frequently start and stop. The database is the foundation of almost any service, so plan it together with scaling and availability.

ElastiCache: Redis as a service

A cache is fast temporary storage where you put an already-computed result so you don't have to compute it again. Redis is the most popular engine for this. ElastiCache is managed Redis (and its open-source fork Valkey): AWS configures the cluster, replicas, and failover.

You use it the same way as ordinary Redis:

• Caching responses with a time to live (TTL, time-to-live — how many seconds until the entry disappears on its own).
• Distributed locking — so two copies of an application don't perform the same operation twice.
• Rate limiting — "no more than 100 requests per minute from one client."

What doesn't go away: discipline in key naming and thinking through TTLs, choosing a serialization format, protecting against a "flood" of simultaneous cache misses (cache stampede) — all of this lives in the code, not in the AWS console. And the main rule for a beginner: ElastiCache is a cache, not a database. Data whose loss is unacceptable (orders, payments) is not stored in it: a cache can drop its contents at any moment.

Queues and brokers: SQS and MSK

When one part of a system wants to hand off work to another without waiting for a response, an intermediary is placed between them — a queue or a message broker. AWS offers two different things for two different jobs.

SQS (Simple Queue Service) is the simplest queue as a service. There is no server, cluster, or maintenance at all: you just put messages in and take them out. Key properties:

• Keeps unread messages for up to 14 days (4 days by default).
• Has a dead-letter queue (DLQ) — whatever couldn't be processed after several attempts ends up there.
• Billing is per request, not per running server.
• Visibility timeout — when a consumer picks up a message, it becomes invisible to others for a set time (from 0 seconds to 12 hours, 30 seconds by default). It's like "renting" a task for the duration of processing: finished in time — you delete it; didn't finish — the message becomes visible again and goes to someone else.

SQS delivery semantics are at-least-once: the same message may arrive again. This isn't a bug but a consequence of reliability — the system would rather retry than lose a message. There's also a FIFO variant (first-in-first-out): it guarantees strict ordering and removes duplicates within a 5-minute window, but pays for it with lower throughput. Living alongside SQS is SNS (Simple Notification Service) — a one-to-many broadcast (pub/sub): one event fans out at once to many subscribers (several SQS queues, functions, and so on).

MSK (Managed Streaming for Apache Kafka) is managed Kafka. It's real Kafka with all its concepts (partitions, consumer groups, retention — how long events are kept in the log), it's just that AWS runs the broker servers and coordination. What MSK does not do for you: topic and partition-key design, consumer idempotency (protection against reprocessing), consumer lag monitoring, a message schema registry.

How do you choose between SQS and MSK? It's a choice between a task queue and an event log. SQS is "take a task, do it, delete it": the message is read and it's gone. Kafka/MSK is a stream of events that many different consumers can read independently, each from its own position, while events stay on disk for the entire retention period. A simple rule: if you just need to hand out work — take SQS, it requires no maintenance at all; if an event has to be digested by several different subsystems or history matters — MSK.

DynamoDB: a different league

DynamoDB is the only item in this list that has no familiar "own" equivalent. It's a managed NoSQL key-value and document database: it returns data in single-digit milliseconds at any scale and is billed per request. But it has a rigid model: access only by key and by indexes you designed in advance, no arbitrary SQL queries like "now show me this instead."

It's not a "NoSQL replacement for PostgreSQL." Designing a DynamoDB table starts with you listing all the queries you'll ever make, and laying out the data to fit them (this is called single-table design). If tomorrow you need to look at the data from a different angle that wasn't in the original list, reworking it is expensive. DynamoDB's honest niche is high-load key-value scenarios with predictable access patterns: user sessions, shopping carts, feature flags, counters. A detailed breakdown is in the separate article about DynamoDB.

Managed vs. your own: how to choose

A sensible default rule: stateful components (databases, caches, queues) go managed until there's a specific reason to do otherwise — for example, you need a PostgreSQL extension that isn't in RDS, or data-residency requirements dictate your own hardware. Before self-managing PostgreSQL when RDS is available, it's worth honestly answering what your engineers will spend the time on that RDS frees up for them.

Where this applies

Managed services are the foundation of almost any application in AWS. A web service almost always sits on RDS or Aurora as its primary storage, puts ElastiCache in front of the database for speed, and decouples slow operations (sending emails, generating reports) through SQS. Event-driven systems, where many subsystems react to changes, are built on MSK.

Typical beginner mistakes:

• Your own server "to save money" without accounting for the cost of engineering time and incidents. Count the total cost of ownership, not just the price of a virtual machine.
• DynamoDB as a universal database. The third rework of single-table design for a new query scenario will cost more than if you'd taken RDS from the start.
• An SQS consumer with no protection against retries. At-least-once delivery means duplicates; processing a money operation twice hurts equally in the cloud and on your own server. Make operations idempotent.
• Default connection pools across many applications eat up the RDS limit. Size the pool based on the database's limit, not on habit.
• Ignoring data-transfer charges. Transferring data between availability zones and through NAT is a noticeable line item on the bill; VPC endpoints and deliberately placing components close to the data help.

What to study next. Understanding where these services live inside the network comes from the article about networking and VPC — managed databases go into private subnets. Who is allowed to reach the data — that's IAM and access management. How to bring the whole system back up after a failure, not just a single database — resilience and recovery. Deeper on specific services: DynamoDB and serverless, where SQS and DynamoDB feel especially natural. To create all this infrastructure through code rather than clicks, start with the fundamentals of infrastructure as code and Terraform. For the big picture of the cloud, the AWS fundamentals and the principles of good architecture in Well-Architected are useful.