Mechanical Sympathy — Know the Machine (Caches, Locality, False Sharing)

Know the machine you run on

Mechanical sympathy (Martin Thompson, borrowing racing driver Jackie Stewart's phrase) is the idea that you don't have to be a hardware engineer, but you write dramatically faster code when you understand how the hardware actually behaves. The dominant fact: not all memory is equal — each level of the hierarchy is orders of magnitude slower than the one above.

The consequences that change how you write code

• Cache lines (64 bytes): memory is fetched a line at a time. Data accessed together should live together. This is why a contiguous array beats a linked list for traversal — the array is prefetched cache-line by cache-line; the list chases pointers all over RAM (cache misses).
• Sequential > random: the same reason B-trees use big pages and logs append sequentially — sequential access rides the cache/prefetcher and the disk's strengths; random access pays the full latency each time.
• False sharing (the concurrency trap): two threads writing two different variables that happen to sit on the same cache line force the line to ping-pong between cores — huge slowdown with no logical contention. Pad hot per-thread fields to separate lines. (Ties directly to Cache Coherence (MESI) & False Sharing.)
• Allocation & GC cost; the latency numbers (see Capacity Estimation) are the intuition you estimate with.

False sharing in code

Two threads increment two different counters, but because the counters sit on the same 64-byte cache line, the line ping-pongs between the cores' caches. The result can be 5–20× slower than the logically equivalent layout with the counters on separate lines.

// BAD: hits and misses share a cache line
class SharedCounters {
    volatile long hits;    // thread 0 increments
    volatile long misses;  // thread 1 increments
}

// GOOD: pad each field to its own cache line
class PaddedCounters {
    @jdk.internal.vm.annotation.Contended  // see the caveat below -- this is NOT a drop-in fix
    volatile long hits;
    @jdk.internal.vm.annotation.Contended
    volatile long misses;
}

Caveat — @Contended is a no-op on application classes by default. The JVM honors it only for JDK-internal classes unless you run with -XX:-RestrictContended (JEP 142), and since JDK 9 the annotation lives in jdk.internal.vm.annotation, which user code can't even see without --add-exports java.base/jdk.internal.vm.annotation=ALL-UNNAMED. Copy the class above without those flags and you get zero improvement and no error — the worst kind of "fix." The portable options: manual padding (seven long fields around each hot field), or use a JDK class that already pads for you — java.util.concurrent.atomic.LongAdder (built on the padded Striped64) is the standard answer for exactly this two-counters workload.

On a typical x86 server with 64-byte cache lines, the unpadded version makes both threads fight for ownership of the same line even though they never touch the same variable. The padded version removes the ping-pong and scales linearly with core count (Martin Thompson, "Mechanical Sympathy" blog; Java @Contended documentation).

The mental model

When something is slow, ask "where does this data live, and am I fighting the cache / the prefetcher / the disk's sequential nature?" Often the fix is a better memory layout, not a better algorithm.

When mechanical sympathy pays — and when it doesn't

This is a last-mile lever, not a first move: get the algorithm and correctness right first, then reach for layout only where a profiler says you are memory-bound. Do not pad every field to its own cache line by reflex — padding trades memory for speed, so on a cold or rarely-contended field it is pure bloat that can push your working set out of the very cache you were trying to respect. The fingerprint that says the layout work is now worth it: a high last-level-cache (LLC) miss rate in perf, or throughput that scales worse than linearly (worse than 1/N per core) as you add cores — the classic false-sharing signature.

Takeaways

• The memory hierarchy is a cliff: L1 ~1ns → RAM ~100ns → disk ~10ms → cross-continent ~150ms.
• Favour contiguous/sequential layouts (arrays, batched I/O); cache lines and prefetch reward locality.
• Watch false sharing in concurrent code; layout can matter more than algorithm.

Drill: three questions this page should let you answer

• "Why does an array traversal beat a linked list even at the same O(n)?"
  Answer: One cache line holds eight 8-byte array elements, so a sequential scan takes one memory stall per eight elements — and the hardware prefetcher hides even those, because the stride is predictable. Each linked-list node is a pointer chase to an unpredictable address: a potential cache miss per element and nothing for the prefetcher to predict. Same complexity class, ~an-order-of-magnitude different constant.
• "Two threads increment two different counters and it's 10× slower than one thread. Diagnose."
  Answer: False sharing — the counters sit on the same 64-byte cache line, so every increment forces the line to ping-pong between cores' caches (MESI ownership transfer) despite zero logical contention. Fingerprint: perf counters show high LLC misses / HITM events on a workload with no shared data. Fix: pad the fields to separate lines or use LongAdder.
• "When is padding the WRONG move?"
  Answer: When the fields are read-mostly (shared read-only lines are cheap — every core keeps a copy in Shared state), when the "hot" pair isn't actually contended (you pay 8× memory per field for nothing, evicting useful data and worsening cache pressure), or when the real problem is true sharing — two threads updating the same variable — which padding cannot fix; that needs sharding the state (per-thread counters aggregated on read).

---

Re-authored for this guide; memory-hierarchy diagram hand-authored as SVG. Follows Martin Thompson's "mechanical sympathy" and the latency-numbers canon. See also: Capacity Estimation (latency numbers), (Concurrency) Memory Model & false sharing, How Indexes Work (pages).

I just read a lesson on **Mechanical Sympathy — Know the Machine (Caches, Locality, False Sharing)** (System Design) and want to truly understand it. Explain Mechanical Sympathy — Know the Machine (Caches, Locality, False Sharing) from first principles using ONE vivid real-world analogy and a visual mental model — draw it as ASCII art or a clear step-by-step diagram — with a concrete example using real numbers. Then ask me one question to check I got the mental picture, and wait for my reply. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

Teach me **Mechanical Sympathy — Know the Machine (Caches, Locality, False Sharing)** interactively. Ask me ONE guiding question at a time, wait for my answer, and adapt to my confusion — build the idea with me step by step instead of explaining it all at once. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

Quiz me on **Mechanical Sympathy — Know the Machine (Caches, Locality, False Sharing)** with 5 questions, easy to tricky, ONE at a time. Tell me if each answer is right; at the end, explain clearly what I got wrong and why. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

Help me remember **Mechanical Sympathy — Know the Machine (Caches, Locality, False Sharing)** for the long term: give the one-sentence intuition, a memorable hook/mnemonic, a tiny worked example, and 3 active-recall flashcards (Q -> A). If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.