What is Leader and Follower Pattern

Background

Distributed systems keep multiple copies of the same data for fault tolerance and to serve more read traffic. But the moment you have several replicas, you face a coordination problem: if any replica can accept a write, two clients can write conflicting values to two different replicas at the same time, and the system has to decide whose write wins. A pure quorum scheme (a write succeeds once a majority of replicas accept it) helps availability, but it doesn't remove the problem — replicas can still receive writes in different orders, and resolving the resulting conflicts (last-write-wins, vector clocks, application-level merge) is extra work most systems would rather avoid.

The pattern

Elect exactly one replica as the leader. Every write goes through it. The leader appends the write to a log and streams that log to the other replicas, the followers, which apply it in the same order. Because there is only one place writes are accepted, there is only one order of writes — no concurrent-write conflicts to resolve. Followers are passive backups that can also serve reads (taking load off the leader) and, if the leader dies, one of them is promoted to take its place.

This is a pattern, not an algorithm: it says “single writer, N passive replicas, promote on failure” without saying how a new leader gets chosen or how the log travels from leader to followers. Real systems fill in those blanks very differently, and conflating the pattern with one specific way of filling them in is the most common misunderstanding of it — see the caveat below before the worked example.

Who elects the leader, and how — a caveat before the example

“Leader election” can mean two very different things in practice, and it's easy to walk away thinking every leader-follower system elects via terms and votes. It doesn't:

• Self-organizing consensus. The replicas themselves run a protocol — Raft, Paxos, ZAB — where nodes vote, track a monotonically increasing term/epoch, and a candidate becomes leader only after winning a majority. No human or external service is involved; failover is automatic and sub-second. etcd, Consul, CockroachDB, and Kafka's controller (KRaft) work this way.
• Externally orchestrated failover. The replicas have no election protocol at all — the leader is just “whichever node is currently marked primary.” An outside control plane (Patroni or repmgr for PostgreSQL, Orchestrator or MHA for MySQL) watches health, decides when to fail over, and reconfigures replication. Classic MySQL primary-replica setups and most Postgres streaming-replication deployments default to this: promotion is a script or an operator running a command, not a vote.

Both are legitimate leader-and-follower systems. The worked example below, and the dedicated Raft lesson that follows this one, use the term-based, Raft-style election because it's the cleanest way to see the mechanics end-to-end — but treat it as one concrete implementation choice, not a defining property of the pattern. A production Postgres cluster fronted by Patroni is just as much “leader and follower” with none of the terms or votes that follow.

Worked example: term-based failover in miniature

Three replicas — A (leader), B, C — replicating an order log. A crashes. (This uses the Raft-style election flagged above; it's one option, not the only one.)

1. B and C stop receiving heartbeats from A. After a randomized election timeout, B times out first, increments the term from 4 to 5, votes for itself, and sends RequestVote(term=5) to C.
2. C hasn't voted in term 5 yet and B's log is at least as up to date as C's, so C grants its vote.
3. B now holds a majority (2 of 3) for term 5 and becomes leader. It starts sending AppendEntries heartbeats, which double as the replication stream and as proof to C that a leader exists for term 5.
4. When A recovers, it rejoins as a follower: it sees term 5 is greater than its own term 4 and steps down immediately, even before it learns it lost an election.

The randomized timeout in step 1 is what keeps two followers from both becoming candidates at the same instant and splitting the vote; the term number is what lets every node agree, without a discussion, which leader is current.

Sync vs. async replication

How the leader ships the log to followers decides what you get when things fail:

• Synchronous — the leader waits for at least one follower to acknowledge before confirming the write. A confirmed write is guaranteed to survive the leader's death, but write latency now includes a round trip to that follower, and if the follower is unreachable, writes stall.
• Asynchronous — the leader confirms immediately and replicates in the background. Lower, more predictable write latency, but a leader crash can lose the last few acknowledged writes that never made it to any follower.

Many systems let you dial this per transaction — MySQL semi-sync replication, Postgres's synchronous_commit — synchronous for writes you can't afford to lose, asynchronous for everything else. Concretely: with a leader in us-east and a follower in eu-west, a client that forces synchronous commit pays roughly the cross-region round trip — approximately 70–80 ms for a typical US-East ↔ EU-West path — on every write, whereas the same write under async acks in single-digit milliseconds locally and the follower simply trails by tens to hundreds of ms. The failover gap has the same shape: automatic Raft-style failover is typically sub-second (a detection timeout of a few hundred milliseconds plus one vote round), while a managed/orchestrated promotion is a multi-second window (commonly ~5–15 s) during which writes are rejected.

When to use it

• Writes need a single, unambiguous order — financial ledgers, inventory counts, anything where two updates landing on different replicas at once is a bug, not a feature.
• Reads vastly outnumber writes and you want to scale read capacity by fanning reads out to followers without touching write semantics.
• You want strong consistency on the write path without hand-rolling conflict resolution.

When not to use it

• Multi-region, write-everywhere workloads. Every write has to reach the leader's region, so clients far from it pay a latency tax on every write, and the region hosting the leader becomes a single point of degraded availability for writes.
• Write availability during a leader failure is not acceptable, even briefly. There is always a detection-plus-election (or detection-plus-promotion) gap — sub-second with Raft, seconds to minutes with a manual or orchestrated failover — during which writes are rejected or queued.
• Clients need to keep writing while offline or partitioned from every replica. The pattern fundamentally requires reaching a leader (or, for consensus-based election, a majority) to accept a write; it will not let an isolated node accept writes.
• Write throughput is bottlenecked by a single node's capacity and the data doesn't shard cleanly — one leader means one node's disk, CPU, and network is your ceiling for that shard.

Leader-follower vs. the alternatives

The short version: leader-follower buys a total order on writes by paying for it with reduced write availability during failures and partitions, and with writes that must travel to wherever the leader happens to be. Multi-leader and leaderless designs buy back that availability by giving up the free ordering — and hand you conflict resolution as homework.

Re-authored from-scratch, drawing on Martin Kleppmann's Designing Data-Intensive Applications (Ch. 5, “Replication”) and Diego Ongaro & John Ousterhout, “In Search of an Understandable Consensus Algorithm” (Raft, 2014); diagrams hand-authored (SVG) for this guide.

Failure trace: the zombie leader and why fencing is required

Failure trace — leader partition with quorum

1. Cluster of 5 nodes; Node L is leader for term 7.
2. Network partitions L away from 3 followers.
3. Followers timeout, elect a new leader in term 8 on the majority side.
4. Old leader L still believes it leads and tries to commit a write.
5. Followers reject the write because their term (8) is higher; if L had a fencing token, the storage layer would also reject the stale term.

Without quorum + fencing, both sides accept writes and data diverges.

The production smell that follows a botched failover is read-your-writes breaking: a client that just wrote through the old leader gets routed to a lagging replica (or to the deposed leader itself) and its own write appears to vanish. Two defenses belong in the design, not the runbook: pin sessions that need read-after-write to the primary, and measure replica lag as a product SLI — treat "promote replica" as a data-loss and dual-writer ceremony, not a button.

Drill ladder

• L1: Why does leader-follower give a total write order "for free"?
• L2: Compare synchronous vs async replication under leader failure.
• L3: In a 5-node cluster, what is the minimum number of nodes that must be reachable to elect a leader?
• L4: How does a fencing token prevent a zombie leader from corrupting data?