Cache Stampede & Invalidation — Thundering Herd and the Hard Half

The cache made things worse — all at once

A single hot key (a trending post, a config) is cached and serving thousands of reads per second from memory. Its TTL expires. Now every in-flight request misses simultaneously and they all query the database with the same query at the same instant — a cache stampede (thundering herd). The DB, sized for the cache-hit rate, gets hit by 1000× its expected load and can fall over, taking the service with it.

Make sure only ONE request rebuilds the key

• Single-flight / request coalescing: the first miss starts the fetch; concurrent misses for the same key wait for that one result instead of each querying. (Go's singleflight, or a per-key in-flight map.)
• Lock + serve-stale: the first miss takes a short lock and recomputes; others briefly serve the stale value rather than pile onto the DB.
• Probabilistic early expiration: refresh the key slightly before the TTL, with random jitter, so popular keys don't all expire on the same tick (this also prevents synchronized expiry across keys).
• Stale-while-revalidate: always serve the cached value; refresh asynchronously in the background. Users never wait on a rebuild.

The other hard half: invalidation

"There are only two hard things in computer science: cache invalidation and naming things." — Phil Karlton

Keeping the cache coherent with the source of truth is the deeper problem. Options trade freshness vs complexity: short TTL (simple, allows staleness), write-through (update cache on write — see Caching strategies), explicit invalidation on change (precise but you must catch every write path, incl. other services), or versioned keys. There's no free lunch — pick the staleness you can tolerate.

Takeaways

• A hot-key expiry triggers a stampede: 1000 simultaneous misses → DB overload → cascade.
• Fix so only one request rebuilds: single-flight, lock+stale, probabilistic early expiry, or stale-while-revalidate.
• Invalidation is the hard half — choose TTL / write-through / explicit / versioned by the staleness you can accept.

---

Re-authored for this guide; stampede diagram hand-authored as SVG. Follows the Memcached/Redis stampede literature and Cloudflare/Instagram engineering. See also: Caching strategies, Designing for Failure (load shedding), Rate Limiting, (Concurrency) the lost-update race.

I just read a lesson on **Cache Stampede & Invalidation — Thundering Herd and the Hard Half** (System Design) and want to truly understand it. Explain Cache Stampede & Invalidation — Thundering Herd and the Hard Half from first principles using ONE vivid real-world analogy and a visual mental model — draw it as ASCII art or a clear step-by-step diagram — with a concrete example using real numbers. Then ask me one question to check I got the mental picture, and wait for my reply. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

Teach me **Cache Stampede & Invalidation — Thundering Herd and the Hard Half** interactively. Ask me ONE guiding question at a time, wait for my answer, and adapt to my confusion — build the idea with me step by step instead of explaining it all at once. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

Quiz me on **Cache Stampede & Invalidation — Thundering Herd and the Hard Half** with 5 questions, easy to tricky, ONE at a time. Tell me if each answer is right; at the end, explain clearly what I got wrong and why. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

Help me remember **Cache Stampede & Invalidation — Thundering Herd and the Hard Half** for the long term: give the one-sentence intuition, a memorable hook/mnemonic, a tiny worked example, and 3 active-recall flashcards (Q -> A). If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.