Designing Dropbox — Chunking, Dedup & Delta Sync, Traced
The insight: never move bytes you don't have to
A file-sync service (Dropbox, Google Drive) looks like "store files," but the hard, defining problem is efficient sync at scale: don't re-upload an unchanged file, don't store the same bytes twice, and propagate a one-line edit without shipping the whole document. The answer is chunking + content-addressable storage.
The architecture
- Chunking: split each file into fixed (~4 MB) blocks.
- Content-addressable storage: hash each chunk (e.g. SHA-256); the hash is its key. Two identical chunks — anywhere, any user — hash the same and are stored once (global dedup).
- Split metadata from blocks: a metadata service stores
file → ordered list of chunk hashes+ versions (a small, transactional store); the chunks live in cheap object storage (S3-like). - Delta sync: on edit, re-chunk, hash, and upload only the chunks whose hash changed — one 4 MB block instead of a 2 GB file.
- Notify & pull: other devices are notified (long-poll / push), then download only the changed chunks.
Traced: you edit one paragraph of a large doc
- Client detects the change, re-chunks the file, hashes each chunk.
- Asks the server which hashes it already has → only chunk
b3is new. - Uploads just b3 to the block store; updates metadata to the new chunk list + version.
- Your other devices get notified, fetch just b3, and reconstruct the file.
The hard parts
- Conflicts: two devices edit offline → versioning + a "conflicted copy" (or CRDTs for mergeable types — see CRDTs).
- Metadata consistency must be strong (it's the source of truth for what a file is); the block store can be eventually consistent.
- Small-file & many-file overhead; compression; encryption (client-side for zero-knowledge).
Takeaways
- Chunk + hash (content-addressable) → global dedup and delta sync; move only changed chunks.
- Separate strongly-consistent metadata from cheap, eventually-consistent block storage.
- Sync = notify → pull changed chunks; conflicts need versioning/CRDTs.
Re-authored for this guide; sync diagram hand-authored as SVG. Complements the "Designing Dropbox" problem page with a traced flow. See also: Distributed File System, content-addressable storage, Replication, CRDTs.
🔨 Practice this hands-on — Design Dropbox / File Sync →
Attempt it from an empty file, break it to feel the failure, then defend it under pushback.
🤖 Don't fully get this? Learn it with Claude
Stuck on Designing Dropbox — Chunking, Dedup & Delta Sync, Traced? Open Claude, copy a block below, and it'll teach you this exact concept — visually and interactively.
🎨 Explain it visually
Build the mental picture, not memorization.
I just read a lesson on **Designing Dropbox — Chunking, Dedup & Delta Sync, Traced** (System Design) and want to truly understand it. Explain Designing Dropbox — Chunking, Dedup & Delta Sync, Traced from first principles using ONE vivid real-world analogy and a visual mental model — draw it as ASCII art or a clear step-by-step diagram — with a concrete example using real numbers. Then ask me one question to check I got the mental picture, and wait for my reply. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.
🤔 Walk me through it (interactive)
Socratic — adapts to where you're stuck.
Teach me **Designing Dropbox — Chunking, Dedup & Delta Sync, Traced** interactively. Ask me ONE guiding question at a time, wait for my answer, and adapt to my confusion — build the idea with me step by step instead of explaining it all at once. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.
🧪 Quiz me & fix my gaps
Active recall exposes what you missed.
Quiz me on **Designing Dropbox — Chunking, Dedup & Delta Sync, Traced** with 5 questions, easy to tricky, ONE at a time. Tell me if each answer is right; at the end, explain clearly what I got wrong and why. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.
🧠 Make it stick
Intuition + hook + flashcards for long-term memory.
Help me remember **Designing Dropbox — Chunking, Dedup & Delta Sync, Traced** for the long term: give the one-sentence intuition, a memorable hook/mnemonic, a tiny worked example, and 3 active-recall flashcards (Q -> A). If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.