This is the final article in the distributed-data series. In the trouble with distributed systems we saw what you cannot trust: the network, the clock, your own process. Here it is about the abstractions that hide all of that, and about consensus — nodes agreeing on a single question. The chapter's punchline is unexpected: half of the problems that look different (leader election, unique name, atomic commit of a transaction) are actually one and the same problem. Let's work our way to it.
Linearizability: the single-copy illusion
In a database with eventual consistency, the same query to two replicas can return different answers. Linearizability (a.k.a. strong consistency) removes this: the system behaves as if there were one copy of the data and every operation were atomic. As soon as a client has written a value, any subsequent reader sees exactly it — this is a recency guarantee, not "it will converge someday."
The classic violation: Alice and Bob watch the final in the same room. Alice refreshes the page, sees the score, shouts to Bob. Bob refreshes — and his request lands on a lagging replica, so the match is "still on." Replica lag became a visible bug because there was a second channel between them (Alice's voice).
Linearizability is not needed everywhere — it is slow (see CAP below) — but in a few places it is critical:
- Leader election. There must be exactly one leader (otherwise split brain). Nodes must agree on who it is.
- Uniqueness constraints. Two people concurrently register one name / book one seat / withdraw from one account — exactly one must win.
- Cross-channel dependencies. The example above: two paths to the data (cache and database, queue and storage) breed a race that linearizability closes.
Don't confuse it with serializability: that is about transaction isolation (several objects, transaction ordering), while linearizability is about the recency of read-write on a single object. They are different guarantees.
The CAP theorem without the myths
Why not make everything linearizable? Because of the network. Picture two datacenters with replication; the link between them broke. A client on the "cut-off" side wants to write. The choice is hard:
- Demand linearizability — the replica on the minority side cannot confirm recency, so it must refuse (become unavailable) until the link is restored. This is "C" — consistency at the cost of availability.
- Want availability — accept the write locally, but the replicas will diverge (non-linearizable). This is "A."
That is CAP. But the popular reading "pick 2 of 3: Consistency, Availability, Partition tolerance" is wrong: a network fault is not something you can "opt out of," it just happens. The honest phrasing: either consistency or availability — when the network is partitioned. With a healthy network you get both. CAP is also narrow — it is about one failure type (a partition) and one model (linearizability), and says nothing about delays or dead nodes. So in practice the theorem is mostly of historical interest.
And a separate fact: surprisingly few systems are linearizable — not for fault tolerance, but for speed. Linearizability is always slow (response time is proportional to network delay), and most systems deliberately don't provide it.
Order, causality, and broadcast
Linearizability implies a total order of operations: a single timeline on which any two operations are comparable. But there is a weaker and often sufficient order — causal order: it orders only operations connected by the happens-before relation. A question must precede an answer; a row must be created before it is updated. Operations that don't know about each other are concurrent and remain incomparable. Causal order is partial: like version history in Git, where branches merge.
Causality can be preserved more cheaply than linearizability. Wall-clock timestamps are useless for ordering (clocks drift), but Lamport timestamps — logical counters that grow on every operation and travel in every message — give a total order consistent with causality. Yet even they are not enough for a unique name: the order is known only after the fact, once all operations are gathered — and you cannot decide "is the name taken right now" that way.
What you actually need is to know when the sequence is fixed. That is total order broadcast: a protocol guaranteeing that all nodes receive the same messages in the same order, reliably, even under failures. This is exactly the replication log: deliver a message = append to the log, and all replicas, replaying it in the same order, converge.
Consensus: it is all one problem
Here is the climax. It turns out the following problems are equivalent — solve one and you get the rest:
- total order broadcast;
- a linearizable compare-and-set register;
- leader election;
- a uniqueness constraint;
- atomic commit of a distributed transaction.
All of them are consensus: making several nodes agree on a single value. And consensus has strict properties (uniform agreement, integrity, validity, and termination — the algorithm must move forward as long as a majority is alive).
Atomic commit is the familiar special case. The classic protocol is two-phase commit (2PC): the coordinator sends "prepare" to all participants (phase 1), collects the "yes" votes, writes the decision to its own log (the point of no return), and sends "commit" (phase 2). Having answered "yes," a participant loses the right to abort on its own — and if the coordinator dies between the phases, the participant is stuck in doubt, holding locks, until the coordinator returns. That is why 2PC is called blocking: the coordinator is a single point of failure. It is a bad consensus.
Good consensus is Raft, Paxos, Zab, VSR. All of them rely on a majority quorum (two majorities cannot both exist → no conflicting decisions) and "epochs": in each epoch there is at most one leader, and a decision is made by a majority vote. The theory scares you with the FLP result (in a fully asynchronous system there is no guaranteed consensus), but in practice partial synchrony or randomization is enough — and consensus is achievable.
The practical takeaway is single and firm: do not implement consensus yourself. Getting it right is nearly impossible. Take a ready-made service — ZooKeeper or etcd: they provide linearizable storage, leader election, distributed locks, and fencing tokens as a service. That is how HBase, Kafka, and Kubernetes work.
Where this applies
Every time the system needs exactly one leader, a unique name, "grab a resource atomically," or a consistent order of events — you have hit consensus, even if you didn't call it that. The practical frame: don't reach for linearizability and distributed transactions without need (they are slow and fragile — applications route around them with sagas and outbox); and where node agreement truly is required, don't invent a protocol, delegate coordination to ZooKeeper/etcd.
Where beginners stumble:
- Confusing linearizability and serializability — one is about the recency of a single object, the other about transaction isolation; different guarantees.
- Reading CAP as "2 of 3" — a network fault is not chosen; the "consistency or availability" choice arises only during a partition.
- Ordering events by a Lamport timestamp and thinking uniqueness is solved — the total order is known only after the fact; you need total order broadcast.
- Writing your own distributed lock / leader election — that is consensus, nearly impossible to get right. ZooKeeper/etcd.
- Pulling 2PC between microservices — a blocking protocol with a coordinator SPOF; in microservices a saga is almost always better.
What to read next: the trouble with distributed systems — partial failures and the fencing token, where consensus begins; replication models — quorums and eventual consistency; distributed transactions — why applications route around 2PC with sagas; ACID and isolation — the serializability not to be confused with linearizability. The primary source is Martin Kleppmann, "Designing Data-Intensive Applications," chapter 9.