CAP Theorem

• Consistency (C) here is linearizability, NOT the 'C' in ACID. It promises: every read sees the most recent committed write, as if there were one single copy. So node B must return the value node A just wrote.
• Availability (A): every request to a non-failing node returns a non-error response in bounded time. Node B must answer, never 'try again later'.
• Partition tolerance (P): the system keeps operating even when the network drops/delays messages between nodes (e.g. A and B can't talk for 5 seconds).

Chain: replication exists → replicas can diverge → to make B's read match A's write, A must ship the update to B and B must wait for it → that waiting is exactly the C-vs-A tension. Cost: people conflate this C with ACID consistency and over-claim 'my SQL DB is CP' without checking what it actually does under a partition.

Chain: client writes x=2 to A → A tries to replicate to B but the link is down → B never learns about x=2 → now B receives the read. B has exactly two choices:

• Return x=1 (the stale value) → it answered, so it stayed Available, but it broke Consistency.
• Refuse / block until the partition heals → it preserved Consistency, but it sacrificed Availability.

There is no third door: while partitioned, B cannot both answer instantly AND answer correctly. Cost / nuance: the theorem is a statement about the partition moment only. The real choice is binary — CP or AP — and it only bites for the seconds/minutes a partition actually lasts.

Chain: any system spread across more than one machine communicates over a network → real networks will drop packets, GC-pause a node for 8s, or sever a switch → so partitions are not optional, they are a fact you must plan for. 'Not tolerating partitions' just means 'when a partition happens, I have no defined behavior' — which in practice degrades to either stale reads (AP) or errors (CP) anyway.

So for any distributed system, P is mandatory; the genuine dial is C vs A. Cost: the only true 'CA' system is a single node (no network = no partition), e.g. one Postgres instance — but that gives up the very scalability/fault-tolerance you replicated for. Saying 'CA' signals you think partitions are avoidable; they aren't.

Chain: a write needs majority acknowledgment (3 of 5) before it commits → the 2-node minority partition can't reach a majority → it rejects writes (and linearizable reads) to avoid serving stale data → the 3-node majority side keeps committing because it still has quorum.

Result: the system stayed Consistent by making the minority side Unavailable for the partition's duration. Concretely, a client hitting the minority gets errors for, say, the 5–30s until the partition heals or a new leader is elected.

When to use: coordination/locks, leader election, config, financial ledgers — anywhere a stale read is dangerous. Cost: you trade uptime for correctness; if no side can reach quorum, the whole cluster halts. That is the point, not a bug.

Chain: AP stores accept the write on whatever replica is reachable → so both sides succeed and the key now has two conflicting versions → when the partition heals, replicas reconcile via gossip/anti-entropy and discover the conflict → they resolve it with a rule: last-write-wins by timestamp (Cassandra) or application-level merge via vector clocks / CRDTs (Dynamo, Riak shopping carts merge by union).

This is eventual consistency: given no new writes, all replicas converge to the same value within a bounded window (often tens of ms, longer under partition).

When to use: shopping carts, social feeds, metrics, session stores — high write availability matters more than a momentary stale read. Cost: last-write-wins can silently drop the price=10 write; the app must tolerate stale reads and design merge logic, pushing complexity up into your code.

Chain: CAP only describes the partition moment, but the vast majority of the time there's no partition → yet you still face a choice every request: wait for more replicas (stronger consistency, higher latency) or answer from one replica (lower latency, possible staleness).

PACELC captures both regimes: if Partition → choose A or C; Else (normal operation) → choose Latency or Consistency.

• DynamoDB / Cassandra default: PA/EL — available under partition, low-latency otherwise.
• etcd / Spanner: PC/EC — consistent under partition AND willing to pay latency normally.

Concrete cost: Spanner's strong consistency leans on TrueTime and adds a few ms of commit-wait even with no partition — you pay latency on every write for correctness you rarely 'see' exercised. Mentioning PACELC shows you know CAP isn't the whole story.

Chain: a write isn't acked until W replicas confirm it; a read polls R replicas and takes the newest version → if R + W > N, the read set and write set must overlap on at least one replica → that overlapping replica is guaranteed to hold the latest acked write → so the read sees it. With N=3, R=2, W=2: 2+2=4 > 3, overlap guaranteed → strong-ish consistency.

• R=1, W=1 → 1+1=2, NOT > 3 → no guaranteed overlap → fastest and most available (any single replica can serve), but you can read stale data.
• W=3 (write all), R=1 → 3+1=4 > 3, fresh reads from one node, but a single slow/down replica blocks every write (availability hit).

Insight: CAP isn't one global switch — you can dial C-vs-A per operation. Note quorum overlap guarantees you see the latest acked value, not full linearizability (no global ordering of concurrent writes). Cost: higher R+W means more nodes must respond → higher latency and lower availability; you're literally trading p99 latency for freshness on each query.

Chain — match the data's tolerance for staleness to the side:

• Payments / ledger → CP. A stale balance lets a user double-spend → correctness is non-negotiable → choose CP (e.g. Spanner, a quorum SQL store); accept that a partition may reject some writes (better a failed payment than a wrong one).
• Add-to-cart → AP. A user must always be able to add items, even mid-partition → choose an AP store that stays writable and reconciles on heal (e.g. DynamoDB/Cassandra, merging carts by union so no item is lost); accept that a stale cart for a few seconds is harmless.

The winning move: don't pick one side for the whole system — split it per data type and name the store for each. Cost / nuance: two stores means two consistency models to reason about and a reconciliation seam between them (e.g. cart → order → ledger), so you must define what happens if the AP side and CP side disagree at checkout.