The Retry Pattern A Solution to Unreliable External Resources

The retry pattern re-issues a failed request after waiting a delay that grows exponentially with each attempt and is randomised (jittered), so a single client rides through a transient fault while a struggling dependency gets progressively more room to recover — and a whole fleet of clients does not re-synchronise into a self-reinforcing traffic wave. That one sentence hides three separate decisions: which errors to retry, how long to wait between attempts, and whether the operation is safe to repeat at all. Get any of the three wrong and retries turn a small blip into an outage.

The which is the easy part and the original instinct is right: retry only faults that are plausibly transient — network timeouts, connection resets, HTTP 429/503, a database deadlock (the loser is told to retry). Fail fast on faults that are deterministic for this exact request: HTTP 400 (bad input), 401/403 (auth), 404, 422. Retrying those just repeats a guaranteed failure and adds load. A retry policy is therefore a predicate on the error, not a blanket catch. The hard parts — the delay math and the safety precondition — are where most engineers get burned, and they are what the rest of this page is about.

The delay formula: exponential backoff + jitter

Waiting a fixed delay (retry every 1s) is the classic mistake: it does nothing to relieve a load-related fault, and it causes every client that failed at the same instant to retry at the same instant — forever, in lockstep. Exponential backoff fixes the first problem by widening the gap each attempt:

window(n) = min(cap, base · 2^n)   // n = 0, 1, 2, ...

With base = 100 ms and cap = 2 s, the windows grow 100 → 200 → 400 → 800 → 1600 → 2000 (capped). The cap stops the delay from ballooning to minutes; a bounded max-attempts (or an overall deadline budget) stops it from retrying forever.

Exponential backoff alone still leaves clients synchronised — they just collide at 100 ms, then 300 ms, then 700 ms. Jitter breaks the synchronisation by randomising the actual sleep inside the window. The three standard recipes (from the AWS Architecture Blog, "Exponential Backoff And Jitter"):

• Full jitter — sleep = random(0, window). Maximum spread; the default choice.
• Equal jitter — sleep = window/2 + random(0, window/2). Keeps a guaranteed minimum gap while still spreading.
• Decorrelated jitter — sleep = min(cap, random(base, prev_sleep · 3)). Uses the previous sleep as the seed, giving a smoother random walk that AWS found best minimises both total work and completion time.

The diagram below shows why the randomisation, not the exponential growth, is what actually saves the downstream service.

A worked trace: retrying a card charge

An order service posts a $49.99 charge to a payment gateway. Policy: base = 100 ms, cap = 2 s, max = 5 attempts, per-try timeout = 2 s, full jitter, retry only transient errors, and — critically — the same idempotency key chg_order_8842 on every attempt. The gateway is briefly overloaded, then a response gets lost on the network.

(Attempt 3's read timeout at 2331 ms is 331 ms start + the 2 s per-try timeout; note the 2 s read timeout and the 2 s backoff cap are independent knobs that merely coincide here.)

The card was charged exactly once, even though the client sent four requests and never got a response to the one that actually succeeded. Now delete the idempotency key: attempt 4 would look like a brand-new charge, and the customer is billed $49.99 twice. That is the central danger of blind retries — a timeout is not a failure, it is an unknown. The request may have fully succeeded; you simply did not hear back.

Idempotency: the precondition blind retries ignore

An operation is idempotent if doing it N times has the same effect as doing it once. Retries are only safe on idempotent operations. GET/PUT/DELETE are naturally idempotent; a raw POST that creates a resource or moves money is not, so you make it idempotent with an idempotency key: the client generates a unique key per logical operation and sends it on every attempt; the server stores key → result the first time and replays the stored result for any repeat. Stripe, PayPal, and AWS all expose exactly this. The same idea inside a system is the idempotent consumer (dedupe by message ID) for queues.

Below, the same correct policy in Java and Go — transient-only, one key, budget- and deadline-aware.

// Java: full-jitter exponential backoff. Retries ONLY transient faults,
// reuses ONE idempotency key, and is capped by a retry budget.
int maxAttempts = 5;
long baseMs = 100, capMs = 2000;
for (int attempt = 0; attempt < maxAttempts; attempt++) {
    try {
        return client.charge(request, idemKey);   // same key on every attempt
    } catch (TransientException e) {               // 5xx / timeout / deadlock
        boolean last = attempt == maxAttempts - 1;
        if (last || !budget.tryAcquire())          // budget stops retry storms
            throw e;
        long window = Math.min(capMs, baseMs << attempt);   // base * 2^attempt
        long sleep  = ThreadLocalRandom.current().nextLong(window + 1); // [0, window]
        Thread.sleep(sleep);
    }
    // PermanentException (400 / 401) is NOT caught -> it propagates immediately
}
throw new IllegalStateException("unreachable");

// Go: same policy — transient-only, single idempotency key, budget + deadline aware.
const maxAttempts = 5
base, cap := 100*time.Millisecond, 2*time.Second
var lastErr error
for attempt := 0; attempt < maxAttempts; attempt++ {
    resp, err := client.Charge(ctx, req, idemKey) // same idemKey every attempt
    if err == nil {
        return resp, nil
    }
    if !isTransient(err) {              // 400 / 401 / validation -> give up now
        return nil, err
    }
    lastErr = err
    if attempt == maxAttempts-1 || !budget.Allow() {
        break
    }
    window := time.Duration(1<<attempt) * base
    if window > cap {
        window = cap
    }
    sleep := time.Duration(rand.Int63n(int64(window) + 1)) // full jitter [0, window]
    select {
    case <-time.After(sleep):
    case <-ctx.Done():                 // respect the overall time budget
        return nil, ctx.Err()
    }
}
return nil, lastErr

Why the naive version is wrong

The tempting version — while (true) { try return call(); catch (Exception e) { sleep(1000); } } — is wrong four ways at once: (1) it loops forever with no cap; (2) it retries every exception, including 400/401 that will never succeed; (3) the fixed 1s delay synchronises all clients into the retry storm the diagram shows; and (4) with no idempotency key it double-charges on the lost-response case in the trace above. Each fix in the code — bounded attempts, a transient predicate, jittered exponential backoff, and a shared key — closes one of those holes.

Retry budgets and the retry-storm multiplier

Retries are dangerous because they multiply through a call chain. If the edge service retries 3× and it calls a service that also retries 3×, one user action can hit the deepest dependency 9 times; across a chain of depth N where each tier retries r times, worst-case amplification is rⁿ. The moment a shared dependency slows down, every layer starts retrying, load spikes exactly when the system is weakest, and the slowdown becomes a full outage. This is retry amplification, and it has caused real cascading failures.

Two disciplines contain it. First, retry at only one layer — usually the one closest to the fault (or the outermost edge) — and let inner calls fail fast. Second, add a retry budget: cap retries as a fraction of successful traffic (gRPC and Finagle default to ~10–20%) using a token bucket, so retries can absorb a blip but can never exceed a small multiple of normal load. That is the budget.tryAcquire() / budget.Allow() gate in the code. When the budget is exhausted, you stop retrying — which is exactly the hand-off point to a circuit breaker.

Pitfalls

• Retrying non-idempotent writes. The double-charge in the trace. If you cannot make it idempotent, do not retry it — surface the ambiguity to the caller instead.
• Treating a timeout as a clean failure. A timeout means "outcome unknown"; the write may have landed. Only an idempotency key makes retrying it safe.
• No cap and no jitter. Unbounded fixed-delay retries are the textbook retry storm — every failed client hammers the recovering service in lockstep.
• Retrying at every layer. rⁿ amplification. Retry once in the chain; budget it everywhere.
• Ignoring Retry-After. A 429/503 often carries a Retry-After header — honour it instead of your own backoff; the server is telling you when it will be ready.
• Retrying 4xx. 400/401/403/404/422 are deterministic for that request; retrying wastes load and hides the real bug.
• Retry blowing the caller's deadline. Backoff sleeps count against the end-to-end SLA; without a deadline/`ctx` check you turn a fast failure into a slow one and the caller times out anyway.
• Retrying on a dead connection or stale DNS. Reusing the same broken socket just re-fails; re-resolve/reconnect between attempts.

When to use it — and when to reach for something else

Reach for retries when the failure signal is transient and independent: sporadic timeouts, 429/503, connection resets, deadlock losers — a fault where "try again in a moment" has a genuinely good chance of hitting a healthy instance. Concrete green lights: error rate is low and spiky (not sustained), the operation is idempotent (or you can add a key), and there is time in the deadline budget to wait.

Trade-offs versus the named alternatives:

• Circuit breaker. Retries assume the fault is brief and keep trying; a breaker assumes a sustained fault and stops trying to let the dependency recover. What retries cost that a breaker saves: wasted work and added load during a real outage. They are complements, not rivals — retry the individual blip, and wrap the retrier in a breaker so that when failures persist you trip open and fail fast. Choose retries when errors are rare and transient; add/trip a breaker when the error rate stays high.
• Fail-fast. Zero latency added, zero amplification risk, but no resilience to blips. Prefer fail-fast for read-only, latency-critical paths where a stale/empty answer is fine and the caller can retry at its own layer.
• Graceful degradation (cache / default / fallback). Returns something immediately instead of waiting; costs you freshness/correctness. Prefer it when a slightly stale answer beats a slow-but-fresh one — e.g. serve the last-known price rather than block on a struggling pricing service.
• Asynchronous queue / outbox. Instead of retrying inline, enqueue the work and let a consumer retry with backoff off the request path. Costs eventual consistency and infrastructure; buys you unlimited-duration retries without holding a caller's connection. Prefer this for writes that can complete out-of-band (emails, webhooks, ledger postings).

Rule of thumb a senior engineer applies: retry inline for sub-second transient faults on idempotent calls, behind a budget and a breaker; degrade or fail-fast on the read path; push durable writes to a queue. Blind unbounded retries on a non-idempotent call is never the answer.

Takeaways

• The mechanism is three decisions, not one: which errors (transient predicate, not blanket catch), how long (capped exponential backoff + jitter, bounded attempts), and is it safe to repeat (idempotency).
• Jitter — not the exponential growth — is what prevents the retry storm; full jitter (random(0, window)) is the safe default.
• A timeout is an unknown, not a failure; only an idempotency key makes retrying a write safe. Skipping it double-charges.
• Retries multiply through a call chain (rⁿ). Retry at one layer, cap it with a retry budget, and wrap it in a circuit breaker so sustained faults fail fast.

Re-authored and deepened for this guide. Sources: Marc Brooker & the AWS Architecture Blog, "Exponential Backoff And Jitter"; the AWS Builders' Library, "Timeouts, retries, and backoff with jitter"; Michael Nygard, Release It! (2nd ed.) on retry storms and circuit breakers; the Stripe and AWS API idempotency-key documentation; and the gRPC / Finagle retry-budget designs.