Idempotency & “Exactly-Once Is a Myth”

"Exactly-once delivery" does not exist — so stop chasing it

Over an unreliable network (fallacy #1), a sender that gets no ack can't know if the message was lost or the ack was lost. So you get one of two guarantees: at-most-once (don't retry — may lose) or at-least-once (retry — may duplicate). There is no exactly-once delivery. The practical answer the whole industry uses:

at-least-once delivery + idempotent processing = effectively-once. Stop trying to deliver exactly once; make duplicates harmless.

What makes an operation idempotent

Doing it twice has the same effect as once. PUT user.email = x, DELETE id=7, SET balance = 100 are naturally idempotent. balance += 50, "append", "send email", "charge card" are not — a retry doubles them. Two ways to fix the non-idempotent ones:

• Idempotency key: the client generates a unique key per logical operation and sends it on every retry. The server records "key → result"; if it sees the key again, it returns the stored result without re-doing the work (the diagram). Stripe's API works exactly this way.
• Natural dedup: a unique constraint (e.g. a transfer id) so the second insert fails harmlessly.

// server, processing a charge with an idempotency key — atomically
if (store.putIfAbsent(key, PENDING) != null)   // someone already has this key
    return store.awaitResult(key);             // return the first attempt's result
Result r = chargeCard(amount);                 // do the real work once
store.put(key, r);
return r;

Where this shows up everywhere

Payment APIs, message-queue consumers (Kafka redelivers on failure → consumers must be idempotent), webhook receivers, and every retry from the Designing-for-Failure lesson. "Kafka exactly-once" is really idempotent producers + transactional offsets — effectively-once, not magic.

Pitfalls

• The dedup check-then-act is itself a race — use an atomic putIfAbsent / unique constraint (the counter++ lesson again).
• Key scope & TTL: keys must be unique per logical op and retained long enough to catch retries.
• Retrying a non-idempotent op without a key = double charge — the classic production incident.

Takeaways

• Exactly-once delivery is impossible; aim for at-least-once + idempotency = effectively-once.
• Make non-idempotent ops safe with an idempotency key (atomic dedup) or a natural unique constraint.
• Mandatory wherever you retry: payments, queue consumers, webhooks.

---

Re-authored for this guide; idempotency-key diagram hand-authored as SVG. Follows the Stripe idempotency docs and Kafka delivery-semantics. See also: The 8 Fallacies (#1), Designing for Failure (retries), Kafka, (Concurrency) Race Conditions, Ticketmaster.

I just read a lesson on **Idempotency & “Exactly-Once Is a Myth”** (System Design) and want to truly understand it. Explain Idempotency & “Exactly-Once Is a Myth” from first principles using ONE vivid real-world analogy and a visual mental model — draw it as ASCII art or a clear step-by-step diagram — with a concrete example using real numbers. Then ask me one question to check I got the mental picture, and wait for my reply. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

Teach me **Idempotency & “Exactly-Once Is a Myth”** interactively. Ask me ONE guiding question at a time, wait for my answer, and adapt to my confusion — build the idea with me step by step instead of explaining it all at once. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

Quiz me on **Idempotency & “Exactly-Once Is a Myth”** with 5 questions, easy to tricky, ONE at a time. Tell me if each answer is right; at the end, explain clearly what I got wrong and why. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

Help me remember **Idempotency & “Exactly-Once Is a Myth”** for the long term: give the one-sentence intuition, a memorable hook/mnemonic, a tiny worked example, and 3 active-recall flashcards (Q -> A). If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.