Skip to article
ALGORITHMICSSystem Design
System Design8 min read

Consensus and Raft

Getting a majority to agree on one value — and the two rules that make it safe.


Five servers must agree which one is the leader, or what the next entry in a log is, while machines crash and messages are delayed.

This sounds like it should be easy and it is the hardest problem in distributed systems. Paxos, the first correct solution, is famous for being nearly impossible to understand. Raft was designed in 2013 with understandability as its explicit goal, and it is what almost everything uses now.

What consensus has to guarantee

Agreement — no two nodes decide differently. Validity — the decided value was proposed by somebody. Termination — a decision is eventually reached.

Raft, in three ideas

One leader at a time. All writes go through it. This is what makes Raft comprehensible: most of the time it is just a leader appending to followers.

Terms. Logical time, incremented on every election. Every message carries a term; a node seeing a higher term immediately steps down. This is how a returning old leader is neutralised without anyone having to detect it.

A replicated log. Entries are appended in order, and an entry is committed once a majority have it.

1 / 5

term 1

A leader log 3
B follower log 3
C follower log 3
D follower log 2
E follower log 2

A is leader for term 1. It has replicated entry 3 to a majority (A, B, C), so entry 3 is committed.

Elections

A follower that stops hearing heartbeats becomes a candidate: it increments the term, votes for itself, and asks the others.

A candidate wins with a majority — 3 of 5. That is the entire safety mechanism for split-brain, because two majorities of the same set must overlap in at least one node, and that node will not vote twice in one term.

Election timeouts are randomised (typically 150–300 ms). Without that, all followers time out together, all become candidates, all split the vote, and the term repeats forever. One line of randomness turns a livelock into a fast resolution.

Two rules do all the safety work

Election restriction. A voter refuses a candidate whose log is less up-to-date than its own.

Commit rule. A leader may only mark entries from its own term as committed by counting replicas.

What it costs

Every write needs a round trip to a majority. In one datacentre that is ~1 ms; across regions it is 50–100 ms, and it is a floor you cannot optimise below.

The leader is a write bottleneck by design. Scaling writes means several Raft groups over different data — which is exactly what CockroachDB, TiKV and Spanner do, one group per range.

When to use it

Use it for metadata: configuration, cluster membership, leases, locks, leader election. Small, critical, low-volume data where correctness is worth a round trip. This is what etcd, Consul and ZooKeeper are for.

Do not use it for bulk data. Every byte crosses a majority.

Do not implement it. Raft is understandable and still very hard to get right — membership changes, log compaction and snapshot transfer are where the production bugs live. Use etcd, or a library like hashicorp/raft or tikv/raft-rs, that has had those bugs found already.