Designing Twitter's Timeline — Fan-out on Write, Traced

Why Twitter's timeline is a caching problem, not a query problem

Reads dwarf writes: people scroll far more than they tweet. Building each home timeline by querying "all tweets from everyone I follow, sorted by time" at read time would hammer the database for every scroll. The design flips it — precompute each user's timeline when tweets are posted.

Traced: posting a tweet (fan-out on write)

1. Write the tweet to durable storage (returns fast).
2. Enqueue a fan-out job (async — the user isn't blocked).
3. Workers look up the author's followers and push the tweet id into each follower's timeline cache (e.g. a Redis list per user).

Traced: loading the home timeline

1. Read your precomputed timeline cache — O(1), the whole point.
2. Merge in tweets from anyone on the pull path (celebrities), sorted by time.
3. Hydrate tweet ids → full tweets (from cache/store).

The celebrity problem & the hybrid fix

A user with 50M followers would trigger 50M cache writes per tweet — a fan-out storm that swamps the system and delays everyone. Fix: don't fan-out celebrities. Their followers pull celebrity tweets at read time and merge them in. Push for the many, pull for the few — the hybrid model real systems use.

Back-of-envelope (see Capacity Estimation): ~6K tweets/s with avg ~100s of followers = millions of cache writes/s — which is exactly why fan-out is async and celebrities are excluded.

Pitfalls

• Synchronous fan-out blocks the poster — always async via a queue.
• Unbounded celebrity fan-out → storm; cap with the pull path.
• Cache/store consistency: timeline holds ids; deleted/edited tweets must reflect on hydrate.

Takeaways

• Read-heavy → precompute (fan-out on write); reads become O(1) cache lookups.
• Fan-out is async (message queue); celebrities use fan-out on read — the hybrid.
• This is caching + queues + estimation applied together.

---

Re-authored for this guide; fan-out diagram hand-authored as SVG. See also: Caching, Messaging System, Capacity Estimation. (Complements the existing "Designing Twitter / Newsfeed" problem pages with a traced flow.)

I just read a lesson on **Designing Twitter's Timeline — Fan-out on Write, Traced** (System Design) and want to truly understand it. Explain Designing Twitter's Timeline — Fan-out on Write, Traced from first principles using ONE vivid real-world analogy and a visual mental model — draw it as ASCII art or a clear step-by-step diagram — with a concrete example using real numbers. Then ask me one question to check I got the mental picture, and wait for my reply. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

Teach me **Designing Twitter's Timeline — Fan-out on Write, Traced** interactively. Ask me ONE guiding question at a time, wait for my answer, and adapt to my confusion — build the idea with me step by step instead of explaining it all at once. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

Quiz me on **Designing Twitter's Timeline — Fan-out on Write, Traced** with 5 questions, easy to tricky, ONE at a time. Tell me if each answer is right; at the end, explain clearly what I got wrong and why. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

Help me remember **Designing Twitter's Timeline — Fan-out on Write, Traced** for the long term: give the one-sentence intuition, a memorable hook/mnemonic, a tiny worked example, and 3 active-recall flashcards (Q -> A). If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.