Chat Multi-Device Sync, Presence Fanout & Server Discovery, Traced

The three parts of a chat system that are actually hard

WebSocket versus long polling is the famous part of a chat design, and it is the easy part. Once you have a persistent connection, three problems remain that no amount of transport choice solves:

• Which server should this client connect to? Connections are stateful, so this is not a load balancer's job.
• The same user has a phone and a laptop. A message must arrive on both, and each may be behind by a different amount.
• The green dot. Naively implemented, it flickers constantly and costs more traffic than the messages do.

Service discovery: picking a chat server

Because a chat connection is long-lived and stateful, the client must be told which chat server to hold a socket to — based on geographic proximity, server capacity, and similar criteria. Apache ZooKeeper is the usual solution: all available chat servers register themselves, and it picks the best one for a given client. The traced login flow:

1. User A tries to log in.
2. The load balancer sends the login request to the API servers.
3. The backend authenticates the user; service discovery selects the best chat server — say server 2 — and returns that server's info to User A.
4. User A connects to chat server 2 over WebSocket.

Note the division of labour: the stateless login request goes through a normal load balancer; the stateful socket goes to a specifically named server. This is also the recovery path — if a chat server holding hundreds of thousands of connections dies, service discovery hands its clients a new server to reconnect to.

Trace: a 1-on-1 message, and why the queue is in the middle

1. User A sends a message to chat server 1.
2. Chat server 1 obtains a message ID from the ID generator.
3. Chat server 1 puts the message on the message sync queue.
4. The message is stored in a key-value store.
5. If User B is online, the message is forwarded to chat server 2, where B is connected. If User B is offline, a push notification is sent from the PN servers.
6. Chat server 2 forwards the message to User B over their persistent WebSocket.

Two design decisions are doing quiet work here. First, the message is persisted before delivery is attempted — so an offline recipient, a dropped socket or a crashed server never loses the message; delivery becomes a retry against durable state rather than a one-shot handoff. Second, the ID comes from a generator, not from the database, and it must be both unique and sortable by time, because that ordering is what makes the sync cursor below possible. This is where the Snowflake-style ID design earns its keep.

Multi-device sync: one cursor per device

A user with a phone and a laptop has two WebSocket connections, potentially to the same chat server. Each device maintains a variable, cur_max_message_id, tracking the latest message ID it holds. A message is new to that device when both conditions hold:

• the recipient ID equals the currently logged-in user ID, and
• the message ID in the key-value store is greater than that device's cur_max_message_id.

With a distinct cursor per device, synchronization reduces to a range query: each device asks for everything above its own watermark. In the diagram the phone at 45 pulls two messages while the laptop at 42 pulls five, from the same inbox, with no coordination between them.

The reason this is the right primitive — and worth understanding rather than memorizing — is what the obvious alternative cannot do. A per-message delivered boolean has one value, so it cannot represent two devices at different positions; you would need a flag per device per message, which grows without bound. The cursor is also idempotent: re-running the pull after a crash or a duplicate wake-up returns the same messages and advances the cursor once, so at-least-once delivery is harmless. Sync becomes a resumable range scan, which is the same trick as a Kafka consumer offset or a database replication log position.

Small group chat: copy into each recipient's inbox

For a group of three (A, B, C), a message from A is copied into each member's message sync queue — one for B, one for C. Think of the sync queue as an inbox per recipient. Each recipient's inbox holds messages from many different senders, so a client only ever reads one place.

This is fan-out-on-write, and it is chosen deliberately for small groups because it simplifies the read path enormously — each client checks only its own inbox — and storing one copy per member is cheap when membership is small. WeChat uses a similar approach and caps groups at 500 members. For large groups, storing a copy per member becomes unacceptable, and you must switch to a shared group log that members read from. The same fan-out-on-write versus fan-out-on-read trade as a news feed, with membership size as the switch.

Presence: the heartbeat, and why the naive version is wrong

Presence servers manage online status over WebSocket. Login writes online status and a last_active_at timestamp to the key-value store; logout flips it to offline. The interesting case is neither.

The naive rule — mark offline on disconnect, online on reconnect — has a real flaw: users disconnect and reconnect constantly in normal use, going through a tunnel, riding a lift, switching from wifi to cellular. Applying the naive rule makes the indicator change far too often, which is worse than being slightly stale, because a flickering green dot is actively misleading.

The fix is a heartbeat: an online client periodically sends a heartbeat event to the presence servers. If a heartbeat arrives within x seconds, the user is online; otherwise offline. In the traced example the client heartbeats every 5 seconds; after three heartbeats it disconnects and does not return within x = 30 seconds, at which point status flips to offline.

That grace window is a deliberate staleness-for-stability trade: you accept showing someone as online for up to 30 seconds after they vanish, in exchange for an indicator that does not flap. Both numbers are tunable and both have costs — a shorter interval means more traffic, a shorter timeout means more flapping.

Presence fanout: a channel per friend pair, and where it breaks

How do A's friends learn about the change? Presence servers use publish-subscribe with a channel per friend pair. When A's status changes, A publishes to channels A-B, A-C and A-D, which B, C and D subscribe to respectively; updates reach clients over WebSocket.

This is elegant for small friend groups and it does not scale, which the design states outright. A group of 100,000 members means each status change generates 100,000 events. Every login by every member multiplies across the membership, so cost grows as members × status-changes — and status changes are frequent because of the very disconnect churn the heartbeat exists to absorb.

The mitigation is to invert the direction: fetch online status on demand — when a user opens a group or manually refreshes the friend list — rather than pushing every change to everyone. Push when the audience is small and the value is immediacy; pull when the audience is large and staleness is tolerable. Presence is one of the clearest cases where the pull model is not a compromise but the correct answer.

Which mechanism, when

Decision	Option	Choose when	Breaks when
Device sync	Per-device cursor on sortable IDs	Always — multiple devices, resumable, idempotent	IDs are not monotonic (random UUIDs make the range query meaningless)
Device sync	Per-message delivered flag	Strictly one device per user	A second device appears — the flag cannot represent two positions
Group delivery	Copy to each inbox (fan-out-on-write)	Small groups (WeChat caps at 500)	Large groups — storage and write cost scale with membership
Group delivery	Shared group log (fan-out-on-read)	Large groups, broadcast channels	Small groups — every read now needs a merge across senders
Presence	Push per friend-pair channel	Small friend lists; immediacy matters	Big groups — 100k members means 100k events per change
Presence	Pull on group open / refresh	Large groups; staleness acceptable	You need instant "typing"-grade immediacy

Pitfalls

• A single shared cursor per user instead of per device. The laptop advances the cursor and the phone never receives the messages — a silent, permanent message loss that looks like a delivery bug.
• Random message IDs. Sync depends on "greater than my cursor"; without time-ordered IDs the comparison is meaningless.
• Marking offline on socket close. Reintroduces the flapping the heartbeat exists to prevent.
• Heartbeat interval equal to the timeout. A single lost heartbeat then marks the user offline; the timeout must span several intervals (5s heartbeat, 30s timeout gives six chances).
• Presence push in large groups. The quadratic-feeling cost usually shows up as presence traffic exceeding message traffic by a wide margin.
• Load-balancing WebSockets round-robin. Connections are stateful and long-lived; a client must be routed deliberately by service discovery, and a "balanced" reconnect storm after a server dies can topple the next server.
• Delivering before persisting. Any crash between send and store loses the message with the sender believing it was delivered.

Cost model — what dominates the bill

A chat system's bill is dominated by something unusual: idle connections and presence chatter, not by messages. Text messages are tiny; the cost is holding millions of sockets open and the heartbeat traffic that keeps them classified.

Rough BOTE for 1 million concurrent users. Each WebSocket needs kernel and application buffers — call it ~10 KB of server memory per connection once buffers and per-connection state are counted — so 1M × 10 KB = 10 GB of RAM purely to hold idle sockets, before any messaging. If one chat server handles ~100,000 connections, that is 10 servers minimum for connection-holding alone, sized by concurrency rather than throughput.

Now presence. At a 5-second heartbeat, 1 million online clients generate 1,000,000 / 5 = 200,000 heartbeat events/second, continuously, forever — almost certainly more events than actual chat messages. And presence fanout is worse: with an average of 100 friends and each user changing status a few times an hour, push-per-friend-pair produces roughly 100 × (a few per hour) × 1M ≈ hundreds of millions of fanout events per hour. That is the line item that surprises people.

Dominant line items: memory/instances for idle connections; then presence heartbeat + fanout event volume; then message storage (cheap — text is small, and the key-value store is the least of your worries).

Levers: lengthen the heartbeat interval (10s instead of 5s halves 200k events/second to 100k, at the cost of slower offline detection); switch large-group presence to pull-on-open, which deletes the fanout line entirely for the biggest groups; and cache messages client-side to avoid re-fetching history, which cuts both bandwidth and key-value store reads.

Operability: the fingerprints of a broken chat system

The distinctive failures here are about state, not throughput. Messages arriving on one device but never on the other is the shared-cursor bug — and the giveaway is that it is permanent, not delayed: the missing messages never appear, because the cursor already moved past them. The same message delivered repeatedly means a cursor that is not advancing after a successful pull, usually because the advance is not persisted before acknowledgement.

Presence indicators flapping points at heartbeat timing — either the timeout is too close to the interval, or heartbeats are being dropped under load, which is worth distinguishing because the second means your presence tier is saturated and is now generating false offline events for healthy users. Presence event volume exceeding message volume by orders of magnitude is the fanout problem, and it appears as a broker or pub-sub bill that nobody can attribute to a feature.

The most dangerous fingerprint is a reconnect storm: when one chat server dies, every one of its ~100,000 clients reconnects at once. Without jittered backoff on the client and admission control in service discovery, those clients land on the next server, overwhelm it, and walk the failure across the fleet — a cascading outage whose root cause looks like "server 2 also failed." Watch also for a chat server whose connection count is far above its peers, which means service discovery is weighting by something stale rather than by live capacity.

Signals worth having: connections per server and their distribution, heartbeat receive rate versus expected (online × 1/interval), presence events published per second split by group size, cursor-advance failures, duplicate-delivery counter, and reconnect rate with backoff-jitter compliance.

---

Re-authored for this guide from the Alex Xu Vol. 1 chat chapter (ZooKeeper service discovery; WeChat's 500-member group cap; 5s heartbeat with a 30s timeout as the illustrative configuration); sync-cursor and heartbeat diagram hand-authored as SVG. Complements the existing "Designing Facebook Messenger" and "Designing WhatsApp/Messenger — Online/Offline Delivery, Traced" pages with the sync, presence-fanout and discovery mechanisms.

I just read a lesson on **Chat Multi-Device Sync, Presence Fanout & Server Discovery, Traced** (System Design) and want to truly understand it. Explain Chat Multi-Device Sync, Presence Fanout & Server Discovery, Traced from first principles using ONE vivid real-world analogy and a visual mental model — draw it as ASCII art or a clear step-by-step diagram — with a concrete example using real numbers. Then ask me one question to check I got the mental picture, and wait for my reply. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

Teach me **Chat Multi-Device Sync, Presence Fanout & Server Discovery, Traced** interactively. Ask me ONE guiding question at a time, wait for my answer, and adapt to my confusion — build the idea with me step by step instead of explaining it all at once. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

Quiz me on **Chat Multi-Device Sync, Presence Fanout & Server Discovery, Traced** with 5 questions, easy to tricky, ONE at a time. Tell me if each answer is right; at the end, explain clearly what I got wrong and why. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

Help me remember **Chat Multi-Device Sync, Presence Fanout & Server Discovery, Traced** for the long term: give the one-sentence intuition, a memorable hook/mnemonic, a tiny worked example, and 3 active-recall flashcards (Q -> A). If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.