Knowledge Guide
Home › Hands-On Builds › SD Design Labs

Design WhatsApp / Chat

Design WhatsApp / Chat

Chat feels deceptively small — "it's just sending a string from phone A to phone B" — and that framing is exactly why it collapses in an interview. A REST API where the recipient polls "any new messages for me?" every second is the obvious first move, and it is wrong in two independent, fatal ways at once: it wastes a staggering fraction of your request budget on empty answers, and it has nothing to say when the recipient's phone is off. This lab derives the design that survives real scale: hundreds of millions of persistent connections, sub-second delivery for online users, durable store-and-forward for offline ones, and per-conversation ordering that holds even when two people's connections land on two different servers. The crux the interviewer is really probing: where does a message live, and how does it find a moving target — a recipient whose connection is pinned to one specific server out of thousands, or who isn't connected at all?

1. The Trap — polling and "fire-and-forget" both break

Trap A: HTTP polling for new messages. The naive real-time story: the client hits GET /messages?since=... on a timer. Pick 1-second polling so chat "feels instant." Now do the arithmetic at scale. Say 250M users are online concurrently (we derive that number in movement 4). Each polls once per second:

250,000,000 online users x 1 poll/sec = 250,000,000 requests/sec

How many carry an actual new message? Chat is bursty and idle-heavy:
a user RECEIVES maybe ~40 msgs spread across a whole day.
  40 msgs/day / 86,400 s = 0.00046 msgs/sec per user
So the fraction of polls that return anything:
  0.00046 / 1 poll/sec  =~ 0.046%  ->  ~99.95% of 250M req/s are EMPTY

You are paying for 250 million requests per second to deliver essentially nothing — each one a full TCP+TLS handshake or at best a kept-alive HTTP round-trip, load-balancer hop, auth check, and DB read that returns "nope." And it is still not instant: worst-case delivery latency is a full poll interval (1s), and if you shorten the interval to 200ms to fix that, you quadruple an already absurd request rate to a billion/sec. Polling makes latency and cost move in opposite directions — you cannot win both. The problem is architectural: the client is asking a question when the server is the one who knows the answer. The server should tell the client, not be interrogated.

Trap B: no offline handling. Suppose you fix real-time with a socket push: sender → server → push straight down the recipient's live connection. Great — until the recipient's phone is in a tunnel, out of battery, or simply backgrounded. There is no live connection to push to. A fire-and-forget push into a closed socket is a message that silently vanishes. WhatsApp's entire product promise is "your message gets there, even if the other person is offline for three days." A design that only works when both parties are simultaneously connected isn't a chat system; it's a walkie-talkie. Offline delivery is not an edge case here — it's the core requirement, and it forces a durable store into the design from the start.

2. Scope it like a senior (ask before you design)

Every one of these answers changes the architecture. Pin them down before drawing a box:

Assume for this lab: 1:1 and groups (small + large); delivery + read receipts; online/last-seen presence; guaranteed offline delivery with per-conversation ordering; at-least-once + client dedup; E2E encryption acknowledged but out of scope; 500M DAU / 40 sent-msgs/user/day / 250M peak concurrent connections.

3. Reason to the design (derive it, don't recite it)

Attempt 1 — request/response REST + polling. Killed in movement 1: the server knows when a message arrives, but a request/response protocol only lets the client initiate, so the client is reduced to guessing-by-polling. The fix is not a faster poll; it's inverting who talks first. We need the server to push. That means a persistent, bidirectional connection the server can write to at any moment — a WebSocket (or MQTT, which WhatsApp actually uses; same shape). One long-lived connection per online client, held open, server pushes down it the instant a message arrives. Polling's 250M empty req/s becomes 250M mostly-idle sockets that cost memory but almost no CPU until a real message flows.

Attempt 2 — one big server holding all the sockets. Now every online client has a live socket — but 250M concurrent connections cannot live on one machine (movement 4 sizes this at ~2,500 machines). So we have a fleet of connection servers (call them WebSocket gateways), and each online user's socket is pinned to one specific gateway — whichever one they happened to connect to. Immediately a new, defining problem appears: when Alice (socket on gateway #417) sends to Bob, which of the 2,500 gateways holds Bob's socket? The sender's gateway has no idea. This is the heart of the whole design.

Attempt 3 — a session registry (the routing crux). We add a fast, shared lookup: a session registry mapping user_id → gateway_id (+ connection handle), updated every time a client connects, disconnects, or migrates. Now the flow is: Alice's gateway receives her message → a message router asks the registry "where is Bob?" → registry says "gateway #912" → router forwards the message to #912 → #912 pushes it down Bob's socket. The registry is a partitioned in-memory store (Redis-like), sharded by user_id, holding one small entry per online user (~250M entries). This stateful routing — a message chasing a connection that is pinned to one box and can move at any instant — is the mechanism that makes chat hard, and it's exactly what separates a real answer from "just use WebSockets."

Attempt 4 — the offline path. The registry lookup has a second answer: "Bob is not connected." Fire-and-forget would lose the message (Trap B). So the router's rule becomes: online → push; offline → write to a durable per-user inbox (a message store / queue). When Bob reconnects, his gateway drains his inbox in per-conversation sequence order before going live. This is classic store-and-forward: the message always lands somewhere durable the instant the recipient isn't reachable, and delivery is retried on reconnect. Persistence is not a "history" feature bolted on — it's the safety net that makes "message can't be lost" true.

Attempt 5 — ordering. Two messages Alice sends to Bob a millisecond apart must arrive in that order, even if #1 goes to Bob's live socket and #2 (Bob just dropped) goes to the inbox, or if they route through different gateways. Wall-clock timestamps won't do it (clocks skew across servers — see the time/clocks concept page). The clean answer: a per-conversation monotonic sequence number assigned by the owner of that conversation's state, and the client orders by (conversation_id, seq), buffering a gap until the missing seq arrives. Per-conversation, not global — because nobody needs, and nobody can cheaply provide, a total order across all of WhatsApp's traffic.

4. Build it — the design worksheet

The artifact you'd produce on the whiteboard. Numbers are traced, not asserted — recompute them before an interview.

Functional requirements

Non-functional requirements

1. Back-of-envelope estimation (BOTE)

2. API / protocol sketch

Over the persistent connection, messages are framed events, not REST calls. A minimal set:

// client -> server (over the socket)
SEND    { client_msg_id, conversation_id, body, ts }
    -> server assigns per-conversation seq, persists, routes
ACK     { conversation_id, up_to_seq }        // "I have everything through seq N"
READ    { conversation_id, up_to_seq }        // read receipt (blue ticks)
PRESENCE_SUB { conversation_id }               // subscribe-on-view

// server -> client (pushed)
DELIVER { conversation_id, seq, sender_id, body, server_ts }
RECEIPT { conversation_id, seq, state: delivered|read, by_user }
PRESENCE{ user_id, state: online|offline, last_seen }

// out-of-band REST (not latency-critical)
GET /conversations/{id}/history?before_seq=...  // paginated backfill
POST /connect  -> issues token; client then opens WSS with it

Note client_msg_id (a client-generated UUID): the sender's own idempotency key. If the sender retries a send after a flaky ack, the server dedups on it — the backbone of at-least-once-without-duplicates.

3. Data model & partitioning

4. High-level diagram — online push vs offline store-and-forward

The sender's message enters through its gateway, hits the message router, which asks the session registry "where is the recipient?". Online (green): forward to the gateway holding the recipient's socket, push it down, ack back. Offline (amber): write to the recipient's durable inbox; on reconnect (dashed) the recipient's gateway drains that inbox in per-conversation seq order. Presence rides the same connection's heartbeat.

WhatsApp chat mechanism: sender pushes a message over its live WebSocket to a gateway; the message router consults the session registry (user to gateway map, sharded by user-id) for the recipient's location; if online the message is pushed down the recipient's connection on its gateway and acked; if offline it is written to the recipient's durable Offline Inbox / message store and drained in per-conversation sequence order when the recipient reconnects; presence rides connection heartbeats.
WhatsApp chat mechanism: sender pushes a message over its live WebSocket to a gateway; the message router consults the session registry (user to gateway map, sharded by user-id) for the recipient's location; if online the message is pushed down the recipient's connection on its gateway and acked; if offline it is written to the recipient's durable Offline Inbox / message store and drained in per-conversation sequence order when the recipient reconnects; presence rides connection heartbeats.

5. Rubric — what a strong answer hits

5. Break it — traced failures

1. Recipient offline — where does the message live, and how is it re-delivered? Alice sends; router queries the registry; answer: "Bob not connected." A fire-and-forget push here (Trap B) drops the message into a closed socket — gone. The fix is the offline path: the router persists the message to Bob's per-user inbox before acking Alice (so Alice's single grey tick means "server has it durably," not "Bob has it"). Bob's phone comes back 3 hours later, opens a socket to gateway #338, which registers him and immediately drains his inbox in (conversation_id, seq) order, pushing each message, then marks them delivered (double grey tick flows back to Alice). The message lived in durable storage the entire time; nothing depended on Bob being reachable at send time. The lesson: the ack boundary is the durability boundary. Accept-then-persist-then-ack, never accept-then-push-then-hope.

2. Ordering across two servers. Alice sends m1 and m2 a millisecond apart. m1 routes to Bob's live socket on gateway #912; between them Bob's socket flaps, so m2 goes to his inbox. Bob reconnects and drains the inbox — if we ordered by arrival or wall-clock, m2 (later, but delivered via a different path) could show before m1, or clocks on #912 vs the store could disagree and scramble them. Because both m1 and m2 got their seq from the single owner of that conversation's shard (m1=seq 51, m2=seq 52) before either was routed, Bob's client renders strictly by seq and holds a gap: if it somehow receives seq 52 first, it buffers it until 51 arrives. The lesson: assign the order at one authority at accept time; never reconstruct it from timestamps at delivery time. (Remove the per-conversation seq and this test fails: interleave two delivery paths and messages reorder.)

3. Connection-server crash — reconnect without losing messages. Gateway #912 holding 100K sockets (including Bob's) hard-crashes. Two problems: (a) those 100K clients' sockets die; (b) the registry still says "Bob → #912," now a lie. Handling: the registry entries are heartbeat-TTL'd, so #912's stale entries expire in seconds; meanwhile a message routed to #912 in the gap fails to push and falls through to the offline inbox (same store-and-forward path as case 1 — the crash is just "offline" from the router's view). Clients detect the dead socket and reconnect (with backoff + jitter to avoid a thundering herd) to a different gateway via the load balancer, re-register in the registry, and drain any inbox backlog. No message is lost because anything the router couldn't push got persisted. The lesson: a gateway crash must degrade to the offline path, and reconnect must be a first-class, jittered, idempotent flow — not an error case.

4. Group message fan-out. Alice posts to a 50-member group. Naive: treat it as 49 independent 1:1 sends from Alice's client — her phone uploads the message 49 times over a phone connection. Terrible on mobile uplink. Better: Alice sends once to the server; the server reads group membership and fans out server-side — push to the online members, write to the inbox of each offline member, all sharing one assigned per-group seq so everyone sees the same order. This is fine at 50. At 100K members it is not (Defend). The lesson: fan-out belongs on the server, and its cost scales with group size — which becomes the next design fork.

6. Optimise — with trade-offs

Each row: what it buys, what it costs, when to pick it — against a named alternative.

DecisionOption AOption BPick when
Real-time transportWebSocket / MQTT (full-duplex persistent)HTTP long-poll · SSE (server-push only)WebSocket/MQTT for chat: full-duplex (client sends AND server pushes on one socket), lowest per-message overhead, MQTT especially battery/bandwidth-frugal on mobile. Long-poll is a fallback where sockets are blocked by a proxy (reconnect churn, higher latency). SSE only fits server→client-only feeds (notifications, live scores) — it can't carry the client's sends, so it's half a chat protocol.
Delivery on sendPush-to-online + store-for-offline (conditional)Always-store, then deliver (store-first for everyone)Push-to-online is the default: online users (the common case) get sub-second delivery without a storage round-trip on the hot path. Always-store is simpler and gives uniform durability semantics but adds storage latency to every message including the online majority — pick it only if you also need full server-side history for every message anyway (then the store write isn't extra work).
OrderingPer-conversation sequence #Global sequence number / wall-clock timestampPer-conversation almost always: it's the only ordering users perceive, and it needs coordination only within one conversation's shard (cheap). A global sequence is a system-wide bottleneck and total order nobody needs; wall-clock ordering is broken by clock skew across servers. Reach for global only for a true system-wide audit log, not user-facing chat.
Group fan-outFan-out on write (push/store per member at send)Fan-out on read (one copy; members pull)Fan-out-on-write for normal groups (≤ a few thousand): delivery is instant and read is cheap. Fan-out-on-read (store the message once per group, members' clients pull on open) wins for very large groups / broadcast (100K+): writing 100K inbox entries per message is untenable, so you flip to pull — the same fan-out-on-write-vs-read fork as a Twitter timeline (celebrity problem). Hybrid: write for small groups, read for huge ones.
PresenceSubscribe-on-view + heartbeat last-seenEager fan-out of every presence change to all contactsSubscribe-on-view always at scale: compute "online" only for the handful of chats a user is actively looking at, and derive last-seen from the connection heartbeat lazily. Eager fan-out costs ~5.8M notif/s (movement 4) — 8× the message load — for green dots. Only eager-fan-out in a tiny team-chat app where contact counts and user counts are small.

7. Defend under drilling

Q1 — "Why WebSocket over long-polling or SSE, concretely?"
Chat needs the server to push unpredictably-timed messages AND the client to send — full-duplex. SSE is server→client only, so it can't carry the client's sends; it's half a protocol. Long-poll simulates push but each message costs a request teardown+setup and adds up to a round-trip of latency and reconnect churn. WebSocket (or MQTT, which WhatsApp uses for its lower header overhead and battery friendliness) holds one persistent full-duplex pipe: idle costs only memory, an active message costs almost nothing. Long-poll stays as a fallback when a corporate proxy blocks WebSocket upgrades.

Q2 — "Walk me through exactly what happens when the recipient is offline."
Router queries the registry, gets "not connected," and persists the message to the recipient's durable inbox before acking the sender (sender sees the single tick = "server has it"). On reconnect the recipient's new gateway registers them and drains the inbox in (conversation_id, seq) order, pushing each message and firing delivered-receipts back. The message is durable from the moment of accept; reachability at send time is irrelevant.

Q3 — "A gateway holding 100K live connections crashes. What do users experience and what do you lose?"
Those 100K clients see their socket drop and reconnect (backoff + jitter) through the LB to other gateways, re-registering in the session registry; the crashed gateway's stale registry entries expire via heartbeat TTL. Any message routed to it during the gap fails to push and falls through to the offline inbox — the same store-and-forward path — so nothing is lost. Cost is a few seconds of reconnection and a load spike on surviving gateways, which is why you keep utilization headroom and jitter reconnects to avoid a thundering herd.

Q4 — "How do you guarantee ordering AND exactly-once delivery?"
Ordering: a per-conversation monotonic seq assigned by the single owner of that conversation's shard at accept time; clients render by seq and buffer gaps, so ordering is immune to multi-path/multi-server delivery and clock skew. Exactly-once: I don't promise true exactly-once — it's effectively impossible across an unreliable network (the classic two-generals problem). I deliver at-least-once (retry on the ack) and make it effectively-once with idempotent dedup: every message carries a stable client_msg_id, so a retried or double-delivered message is deduped by the server (on send) and the recipient client (on receive). The honest senior answer is "at-least-once + idempotency," not "exactly-once."

Q5 — "Presence looks cheap. Is it?"
No — it's the sleeper cost. Eagerly pushing every presence toggle (116K/s) to every contact (~50) is ~5.8M notifications/s, roughly 8× peak message traffic, all for status dots. So presence is subscribe-on-view: compute online-ness only for chats a user is actively viewing, and derive last-seen lazily from the connection heartbeat rather than emitting an event per change.

Q6 — the 100× escalation: "How does a 100,000-member group / broadcast message work?"
Fan-out-on-write breaks here: one message would mean 100K inbox writes + up-to-100K pushes, per message — and a chatty large group multiplies that into millions of ops per second. Flip to fan-out-on-read for large groups: store the message once in the group's own log (keyed by group_id, single per-group seq), notify online members with a lightweight "new in group X" nudge, and have each member's client pull the group log on open. It's the celebrity/fan-out fork from timeline design: write is fine for small groups, read is mandatory for huge ones; run a hybrid keyed on group size. Rate-limit sends into giant groups and consider broadcast-only semantics for the 100K case.

Q7 — "What breaks first at 10× scale (5B DAU-equivalent)?"
Not raw message throughput (partitioned by conversation-id, it scales horizontally). The pressure points: (a) gateway fleet + registry — 2.5B concurrent connections means ~25K gateways and a much larger, hotter registry, so registry sharding and gateway autoscaling dominate ops; (b) presence fan-out — already 8× message load, it scales super-linearly with contacts and is the first thing to melt without subscribe-on-view; (c) large-group fan-out — the read/write fork becomes mandatory, not optional; (d) cross-region — connections and conversation shards span continents, so you pin a conversation to a home region and accept that cross-region delivery adds a WAN hop rather than trying to globally synchronize.

8. You can now defend

Re-authored/Deepened for this guide. Model-answer reading: Designing WhatsApp/Messenger — Online/Offline Delivery, Traced (System Design Problems). Concept cross-links: Long-Polling vs WebSockets vs Server-Sent Events, Introduction to Messaging System (the offline store-and-forward queue), Consistent Hashing (sharding gateways and conversation stores), What Is Message Ordering, How Do Partition Keys Affect It, and When Can Ordering Break, Heartbeat (presence + stale-session expiry), and Distributed & Network Gotchas — WebSocket Reconnect (reconnect on gateway crash).

🤖 Don't fully get this? Learn it with Claude

Stuck on Design WhatsApp / Chat? Open Claude, copy a block below, and it'll teach you this exact concept — visually and interactively.

🎨 Explain it visually

Build the mental picture, not memorization.

I just read a lesson on **Design WhatsApp / Chat** (Hands-On Builds) and want to truly understand it. Explain Design WhatsApp / Chat from first principles using ONE vivid real-world analogy and a visual mental model — draw it as ASCII art or a clear step-by-step diagram — with a concrete example using real numbers. Then ask me one question to check I got the mental picture, and wait for my reply. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.
🤔 Walk me through it (interactive)

Socratic — adapts to where you're stuck.

Teach me **Design WhatsApp / Chat** interactively. Ask me ONE guiding question at a time, wait for my answer, and adapt to my confusion — build the idea with me step by step instead of explaining it all at once. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.
🧪 Quiz me & fix my gaps

Active recall exposes what you missed.

Quiz me on **Design WhatsApp / Chat** with 5 questions, easy to tricky, ONE at a time. Tell me if each answer is right; at the end, explain clearly what I got wrong and why. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.
🧠 Make it stick

Intuition + hook + flashcards for long-term memory.

Help me remember **Design WhatsApp / Chat** for the long term: give the one-sentence intuition, a memorable hook/mnemonic, a tiny worked example, and 3 active-recall flashcards (Q -> A). If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

📝 My notes

← Design a Web CrawlerDesign Dropbox / File Sync →