Memory & Cache-Sizing Playbook

The formula

Cache size = hot_fraction × total_items × bytes_per_item × (1 + overhead)

Overhead (~25–50%) covers keys, pointers, and fragmentation in Redis/Memcached.

The 80/20 rule

Usually ~20% of items serve ~80% of reads. Cache the hot fraction, not everything — that’s what makes caching cheap and effective.

Worked example

500M items, ~1 KB each, ~20% hot, 30% overhead:

• Working set = 0.20 × 500M × 1 KB = ~100 GB × 1.3 ≈ ~130 GB.
• Gate: >~200–300 GB won’t fit one node ⇒ shard the cache (consistent hashing).

Hit-ratio economics

Effective latency = hit% × cache_latency + miss% × db_latency. With a 95% hit ratio: 0.95×0.5ms + 0.05×10ms ≈ 0.98 ms — but your p99 lands in the 5% miss path, so size the cache to keep that miss path survivable.

Hit-rate sensitivity — the DB sees the derivative

DB load = miss% × QPS, so the DB feels changes in (1 − hit), not in hit. At 100K QPS:

Hit rate	DB QPS at 100K frontend	Multiple of the 99% case
99%	1K	1×
95%	5K	5×
90%	10K	10×
~0% (cold restart)	100K	100×

A 9-point hit drop (99% → 90%) is a 10× DB-load jump. This is why a cold cache restart is an outage: hit briefly ~0% → 100K QPS lands on a DB sized for 5K (20× overload) — the reason cache warming, request coalescing, and a DB sized for at least the eviction-storm miss rate (not the steady-state one) are part of cache sizing, not separate topics.

When cache-sizing math misleads

Trap	Wrong conclusion	Fix
Size cache = full dataset	Huge Redis bill, low hit rate if access is skewed	Size for hot working set (often 5–20% of keys, higher % of traffic)
Value size = raw field sum	Underestimate 2–3×	Measure serialized size (JSON/protobuf) + Redis overhead per key (~50–100B+)
Ignore fragmentation / replicas	OOM after deploy	×1.5 fragmentation; ×(1+replicas) for HA memory
No eviction policy stated	Redis maxmemory hits → write errors	allkeys-lru vs volatile-ttl; know which keys may die

Boundary: 50M keys, avg 2 KB serialized, but top 5% keys take 80% traffic. Hot set ≈ 2.5M × 2KB × 1.5 frag × 2 (primary+replica) ≈ ~15 GB, not 50M×2KB×3 ≈ 300 GB for the full set.

When NOT to enlarge cache

If hit rate is already 95% and p99 is still high, the bottleneck is miss path or lock contention — more RAM will not help. Profile miss latency first.

When NOT to cache at all: (1) low reuse / uniform access — fill never amortizes; (2) linearizable reads required after every write; (3) working set already fits DB buffer pool at ≥98% hit — Redis only adds RTT. Example: 8 GB catalog in a 16 GB Postgres buffer pool with 99% buffer hits → skip app cache; contrast 2 TB catalog with 5% hot (100 GB) → cache the hot set.

Ops signals your cache is mis-sized: a falling hit rate, rising evictions/s, a fragmentation ratio above ~1.5, OOM kills, and replica memory that scales with RF — watch these, not just total RAM.

See also: Senior corrections (hot fraction is a curve, not a constant).

---

Formulas are standard/public-domain engineering math. Approach and reference-table format adapted from the System Design Primer (CC BY 4.0), Jeff Dean’s latency numbers, the DesignGurus capacity-estimation guide, and Little’s Law.

I just read a lesson on **Memory & Cache-Sizing Playbook** (System Design) and want to truly understand it. Explain Memory & Cache-Sizing Playbook from first principles using ONE vivid real-world analogy and a visual mental model — draw it as ASCII art or a clear step-by-step diagram — with a concrete example using real numbers. Then ask me one question to check I got the mental picture, and wait for my reply. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

Teach me **Memory & Cache-Sizing Playbook** interactively. Ask me ONE guiding question at a time, wait for my answer, and adapt to my confusion — build the idea with me step by step instead of explaining it all at once. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

Quiz me on **Memory & Cache-Sizing Playbook** with 5 questions, easy to tricky, ONE at a time. Tell me if each answer is right; at the end, explain clearly what I got wrong and why. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

Help me remember **Memory & Cache-Sizing Playbook** for the long term: give the one-sentence intuition, a memorable hook/mnemonic, a tiny worked example, and 3 active-recall flashcards (Q -> A). If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.