System Design Building Blocks

Chain: 200M × 2 = 400M tweets/day → ÷ 86,400s ≈ ~4,600 writes/sec average, and peak is ~2–3x average → ~9,000–14,000 writes/sec (round to ~12K). Reads dominate: if each user refreshes a 100-tweet feed several times/day, you land around ~100K–1M reads/sec.

• This read:write ratio of roughly 100:1 is the single most important number — it is what justifies caching and read replicas later.
• Storage: 400M tweets × ~300 bytes ≈ 120 GB/day → ~44 TB/year of raw text → forces partitioning.

Cause → effect: the estimate dictates which blocks are mandatory — a read-heavy ratio mandates caches/replicas, the storage figure mandates sharding. Cost/trade-off: these are order-of-magnitude guesses, not precise figures — stating a wrong assumption confidently (e.g. forgetting the peak multiplier) makes you under-provision by ~3x. Always say your assumptions out loud so the interviewer can correct them cheaply.

Chain: clients can't know which of 10 servers to hit → an LB sits behind one virtual IP / DNS name → it spreads 12,000 req/sec across 10 boxes = 1,200 each, comfortably under the 2,000 limit. It also runs health checks (e.g. every few seconds) and stops routing to a dead node within seconds, so one crash doesn't drop traffic.

• Mechanism: L4 LBs route on IP/port (fast, payload-opaque); L7 LBs parse the HTTP request and can route /images vs /api differently (smarter, slightly more CPU per request).
• When: the moment you have >1 server, or want zero-downtime rolling deploys.

Cost/trade-off: the LB itself is now a single point of failure — if it dies, all 10 healthy servers become unreachable. Fix: run LBs in an active-passive pair with a floating/virtual IP that fails over to the standby, which adds cost and one more component to monitor. You've traded many per-server failure modes for one smaller, centralized one that you then make redundant.

Chain: round-robin assumes every request costs the same. But if server A draws three 10-second report queries while B/C get 10ms lookups, round-robin keeps feeding A its even share → A's queue grows → its p99 latency spikes while B/C sit idle.

• Least-connections: routes to whoever has the fewest in-flight requests → naturally drains work away from the slow/overloaded A. Best for uneven request durations.
• Weighted round-robin: a 16-core box gets weight 4, a 4-core box weight 1 → handles heterogeneous hardware.
• IP/URL hash: the same client always lands on the same server → enables warm local cache hits / sticky sessions.

Cost/trade-off: least-connections needs the LB to track live connection counts (extra state and bookkeeping). Hash-based stickiness deliberately breaks even distribution: one whale client (a celebrity, a hot key) can overload its assigned server, and naive mod N hashing reshuffles nearly every client when a server is added or removed — which is the exact pain consistent hashing later solves by remapping only ~1/N of keys.

Chain: cache-aside — the app checks Redis first; on a miss it reads the DB, then writes the value back to Redis with a TTL. At a 90% hit rate: 90% of reads cost 1ms, 10% cost 1ms + 20ms ≈ 21ms → weighted average ≈ 3ms vs 20ms before, and crucially the DB now sees only 10% of read traffic (e.g. 100K → 10K reads/sec). That drop is what lets the DB survive.

• p99 effect: the slow tail (DB queueing under load) is now hit by far fewer requests, so once the DB is no longer saturated the p99 collapses.

Cost/trade-off — staleness: a cached value can be out of date until its TTL expires. You pick the dial: short TTL (fresher, more DB load) or write-through / invalidate-on-write (consistent, more write-path complexity). Eviction: RAM is finite, so LRU drops cold keys — only worth it if the hot subset fits in memory. Stampede: when a hot key expires, thousands of concurrent misses hammer the DB at once — mitigate with a per-key rebuild lock (only one request repopulates) or jittered TTLs. Caching trades correctness-by-default for speed.

Chain: partitioning splits data across N nodes so each holds ~44 TB / N and absorbs ~12,000 / N writes/sec. The shard key decides everything downstream.

• Hash(user_id): spreads load evenly → no single hot node. But a query like "all tweets between Jan–Mar" has no single home and must scatter-gather across all N shards, and with naive mod N hashing, adding a node remaps nearly every key (use consistent hashing to limit churn).
• Range by date: time-range queries hit one shard → fast and cheap. But today's shard takes ~100% of writes while older shards sit idle → a write hotspot that moves over time.

Cost/trade-off: the deeper pain is cross-shard operations — a JOIN or a transaction spanning two shards needs distributed coordination (two-phase commit, slow and failure-prone) or app-level stitching. Celebrity hotspot: one user with 100M followers can overwhelm their shard regardless of key choice. Partitioning buys horizontal scale at the price of cheap joins and global transactions — so you pick a key that matches your dominant query pattern.

Chain: an index is a separate sorted structure (typically a B-tree) mapping user_id → row locations. Instead of a full scan (O(n), ~10M row reads), the DB descends the tree (O(log n); log₂(10M) ≈ ~23 levels of comparison, and far fewer actual node reads thanks to high B-tree fanout) → the 10s scan becomes a handful of seeks ≈ ~1ms.

• Mechanism: it trades a linear scan for a logarithmic lookup by keeping the indexed column pre-sorted.
• When: columns you frequently filter / sort / join on AND that are selective (many distinct values).

Cost/trade-off: every index is extra storage and must be updated on every write — each insert/update/delete also rewrites every affected index's B-tree. So 5 indexes can make writes meaningfully slower (often ~2–3x) and bloat disk. Indexing a low-cardinality column (e.g. a boolean) barely helps because it can't narrow the candidate set. Indexes optimize reads by taxing writes — the wrong default for a write-heavy table.

Chain: leader-follower replication — all writes go to the leader, which streams its change log to 2 followers. Now: (1) reads spread across leader + 2 followers → roughly 3x read capacity; (2) if the leader dies, a follower is promoted (failover) → no committed data lost, only a brief unavailability window.

• Mechanism: followers replay the leader's log, trailing it by anywhere from a few ms to seconds under load.

Cost/trade-off — replication lag: with async replication the leader acks the write immediately, so a user who posts (write → leader) and instantly reads from a lagging follower sees nothing yet → a read-your-writes violation. Fixes: pin that user's reads to the leader for a short window after a write, or use sync replication (the leader waits for a follower to confirm — safer, but every write now pays the slower of the two and write latency rises). Replication trades a window of inconsistency for availability and read scale — so you decide per read path how fresh the data must be.