CMD Guide
HomeDatabasesIndexing & Storage

Indexes in Practice — When They're Used (and Ignored)

An index exists — so why is the query still slow?

Having an index doesn't mean the optimizer uses it. The single most useful debugging skill here is knowing when an index applies and when it's silently skipped. The mental model: a B+tree index is a phone book sorted by its columns — you can only seek on a prefix of that sort order.

A composite index on (last, first) sorted like a phone book; you can seek by last name or last+first, but not by first name alone
A composite index on (last, first) sorted like a phone book; you can seek by last name or last+first, but not by first name alone

The leftmost-prefix rule (composite indexes)

An index on (a, b, c) can serve a, a AND b, and a AND b AND c — but never b or c alone, because the data is sorted by a first. Order the columns by what you filter on most (and put equality columns before range columns).

When an index is IGNORED

QueryWhy the index is skippedFix
WHERE LOWER(email)=?function on the column — the index is on email, not LOWER(email)functional index on LOWER(email)
WHERE name LIKE '%son'leading wildcard — can't seek a prefixtrailing wildcard, or a trigram/full-text index
WHERE phone_str = 42 (phone_str is varchar)implicit cast on the column (column is cast to int to match literal, breaking sargability)match the column type: phone_str = '42'
WHERE status = 'active' (90% of rows)low selectivity — a seq scan is cheaper than millions of index hopsnothing — the optimizer is right

Covering index = index-only scan

If the index contains every column the query needs, the DB answers from the index alone and never touches the table (no bookmark hop). E.g. for SELECT total FROM orders WHERE user_id=?, an index on (user_id, total) is covering.

EXPLAIN SELECT total FROM orders WHERE user_id = 7;
--  Index Only Scan using orders_user_total on orders   <- covering, no table access
-- vs Index Scan  (then a heap fetch per row)  vs  Seq Scan  (no usable index)

Dense vs sparse: how many index entries per record?

One more axis worth having in your head when you picture "the index": does it carry an entry for every record, or one per block?

Side-by-side dense vs sparse index over 12 sorted records in 3 disk blocks: the dense index holds all 12 keys and lookup of key 37 follows its pointer directly to the row; the sparse index holds only the first key of each block (10, 30, 40), so lookup of 37 picks the largest entry at most 37, which is 30, jumps to block 2 and scans forward 30, 33, 37; caption notes sparse requires the file to be sorted on the key
Side-by-side dense vs sparse index over 12 sorted records in 3 disk blocks: the dense index holds all 12 keys and lookup of key 37 follows its pointer directly to the row; the sparse index holds only the first key of each block (10, 30, 40), so lookup of 37 picks the largest entry at most 37, which is 30, jumps to block 2 and scans forward 30, 33, 37; caption notes sparse requires the file to be sorted on the key

A dense index keeps one entry per record: to find key 37 you probe the index and follow the pointer straight to the row — direct hit, but the index is as long as the table. A sparse index keeps one entry per block (each block's first key): finding 37 means taking the largest entry ≤ 37 (here 30), jumping to that block, and scanning forward — 30, 33, 37 — so you trade a short in-block scan for an index smaller by a factor of rows-per-block, often the difference between an index that lives in RAM and one that doesn't. The catch is the precondition: sparse only works when the data file itself is sorted (clustered) on the indexed key, because both "largest entry ≤ k" and the forward scan rely on physical order — which is why a secondary index over an unsorted heap must be dense. A B+tree quietly contains both ideas: its leaf level is a dense index (every key appears once), while the internal levels are a sparse index over the leaves — one separator key per child page. That is the whole trick behind its shape: dense at the bottom for exact lookups, sparse above so the top levels stay small enough to cache.

Pitfalls

Takeaways

🎯 Drill Ladder — survive the follow-ups

L0 · An index gets used only when the query's predicate matches its leftmost-sorted prefix AND the optimizer's cost model — driven by live statistics — believes seeking beats scanning.

L1 · ① Concurrency — "what happens to the index when two transactions write the same row at once?"
Trap: "The index is a read-only structure — concurrent writes are the table's problem, not the index's."
Bar: Every UPDATE inserts a new index entry pointing at the new heap version (MVCC) while the old entry lingers until vacuum reclaims it, so a hot row bloats its own index entries before cleanup catches up. Concurrent inserts on a monotonically increasing key (auto-increment id) all target the same rightmost B+tree leaf, serializing on that page's write latch regardless of isolation level. connects-to: MVCC, locking & row versions

L2 · ② Failure — "the server crashes mid-CREATE INDEX CONCURRENTLY. What's on disk?"
Trap: "Indexes are just derived data — recovery replay rebuilds it automatically like everything else."
Bar: Committed row writes replay cleanly from WAL, but an online index build that hadn't finished its second table scan is left marked INVALID — never auto-completed, never auto-healed. It silently drops out of planner consideration until someone notices the regression, drops it, and rebuilds; a naive non-concurrent rebuild instead takes a table lock that stalls every writer. connects-to: How Indexes Work (B+tree internals)

L3 · ③⑥ Scale/Cost — "12 indexes, 5k writes/sec: why did write p99 double and disk triple?"
Trap: "Indexes only add read-path complexity; writes go straight to the table."
Bar: Every INSERT/UPDATE/DELETE must maintain every index its columns touch, so write cost is O(#affected indexes) — 12 indexes means up to 12 extra B+tree page writes (plus WAL) per row change. Disk triples because each index duplicates its indexed columns and accrues its own MVCC dead-entry bloat until vacuum runs, and a covering index that saves read I/O by carrying extra columns makes the write and storage cost worse, not better. connects-to: partitioning & local indexes at scale

L4 · ⑤ Adversary/Edge — "staging shows Index Scan, prod runs Seq Scan on the same query. Why?"
Trap: "I added an index, so it must be used — this has to be an optimizer bug."
Bar: The planner costs plans off stored statistics (histogram/cardinality), not live data; once the matching fraction of rows crosses a threshold, sequential I/O genuinely beats the random I/O of an index scan and the seq scan is the correct plan. Stale stats after a bulk backfill or skipped ANALYZE push that estimate wrong in either direction — the fix is re-running ANALYZE / raising the stats target, not blaming the optimizer. connects-to: the cost-based query optimizer

L5 · ④ Time/Lifecycle (worst case) — "index on (tenant_id, status) worked for a year; six months after 'soft delete' got folded into status, latency spikes only during the nightly batch job."
Trap: "Once an index is tuned and verified with EXPLAIN, it stays correct — the schema didn't change, so the index can't be the problem."
Bar: As soft-deleted rows accumulate, status's cardinality collapses toward one dominant value, so the leftmost-prefix (tenant_id, status) index crosses the selectivity threshold for some tenants but not others, and the planner flips between Index Scan and Seq Scan per tenant depending on whichever stats snapshot was last captured. The nightly batch mass-updates status, invalidating those stats and triggering autovacuum-driven bloat cleanup concurrently with live query traffic, so plan instability and vacuum I/O spike latency at the same moment. connects-to: fanout, optimizer stats & composite indexes

The floor keeps dropping: staff+ perturbation beyond L5 asks you to design so the flip is impossible up front — partial indexes filtered on status, or partitioning on the soft-delete boundary — so each shard's local index sees stable, unimodal cardinality no matter when the batch job runs.

Self-locate: died at L1 → mid-level; L4+ → staff signal.

Facing any new concept? Hit it with the six: concurrent? failing? at 100×? over time? adversarial? worth the cost? — that's the interviewer's whole playbook.


Re-authored for this guide; phone-book/leftmost-prefix diagram hand-authored as SVG. Follows Use The Index, Luke! (Markus Winand). See also: How Indexes Work, How a Query Executes.

🎯 STANDOUT elevation: Why / example / when-not / failure / panel / drills — Indexes in Practice — Used vs Ignored

Why this exists / the decision it encodes

Indexes are sorted phone books. The optimizer uses them only when a predicate matches a leftmost prefix and the cost model (stats-driven) believes seeking beats scanning. Creating an index is not a guarantee — EXPLAIN is. Equality-before-range column order is the design decision that makes or breaks composite indexes.

Worked example with numbers or traced SQL/FD

Index (last, first): seek last='Adams'; seek last+first; NOT first alone
Composite equality-before-range worked:
  Index (status, created_at) for WHERE status='open' AND created_at > '2026-01-01'
  status equality first → contiguous slice; then range on created_at
  Reversed (created_at, status): range on created_at first → status filter becomes residual
Ignored: LOWER(email)=?, LIKE '%son', type cast on column, status matching 90% rows
Covering (user_id, total): Index Only Scan for SELECT total WHERE user_id=?
Write cost: each index is O(1) extra B-tree maintenance per row change

When NOT / named alternative

Do not index every column "just in case" — write amplification and bloat. Do not fight a seq scan on 90% selective predicates — the planner is right. Prefer partial indexes (WHERE status='open') when a hot subset is small. Functional index when you must filter LOWER(email).

Failure mode / ops fingerprint / interview trap

Ops: INVALID index after crashed CREATE INDEX CONCURRENTLY — planner ignores it silently. Staging Index Scan vs prod Seq Scan from different stats. Interview: "I added an index so it must be used" — staff rejects.

Domain judgment (K11 theory-bridge / K12 concurrency / K13 query-judgment)

K13: indexing is pure query/schema judgment under stats and write cost. K12: concurrent writes bloat MVCC index entries; monotonic keys hotspot the rightmost leaf. Connects to the staff drill ladder already on the page.

Hostile-panel drills (with model answers)

Q1. Leftmost-prefix rule for (a,b,c)?
Model answer: Serves a, (a,b), (a,b,c) predicates. Does not serve b alone or c alone as a seek prefix.

Q2. Why put equality columns before range columns in a composite index?
Model answer: Equality on the leading column selects a contiguous segment; a range on a leading column scatters subsequent equality filters into residual checks and weakens the seek.

Q3. Name four reasons an index is ignored.
Model answer: Function on column; leading wildcard LIKE; implicit type cast on column; low selectivity where seq scan is cheaper; also: wrong column order / not leftmost.

🤖 Don't fully get this? Learn it with Claude

Stuck on Indexes in Practice — When They're Used (and Ignored)? Open Claude, copy a block below, and it'll teach you this exact concept — visually and interactively.

🎨 Explain it visually

Build the mental picture, not memorization.

I just read a lesson on **Indexes in Practice — When They're Used (and Ignored)** (Databases) and want to truly understand it. Explain Indexes in Practice — When They're Used (and Ignored) from first principles using ONE vivid real-world analogy and a visual mental model — draw it as ASCII art or a clear step-by-step diagram — with a concrete example using real numbers. Then ask me one question to check I got the mental picture, and wait for my reply. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.
🤔 Walk me through it (interactive)

Socratic — adapts to where you're stuck.

Teach me **Indexes in Practice — When They're Used (and Ignored)** interactively. Ask me ONE guiding question at a time, wait for my answer, and adapt to my confusion — build the idea with me step by step instead of explaining it all at once. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.
🧪 Quiz me & fix my gaps

Active recall exposes what you missed.

Quiz me on **Indexes in Practice — When They're Used (and Ignored)** with 5 questions, easy to tricky, ONE at a time. Tell me if each answer is right; at the end, explain clearly what I got wrong and why. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.
🧠 Make it stick

Intuition + hook + flashcards for long-term memory.

Help me remember **Indexes in Practice — When They're Used (and Ignored)** for the long term: give the one-sentence intuition, a memorable hook/mnemonic, a tiny worked example, and 3 active-recall flashcards (Q -> A). If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

📝 My notes