hard Burn-Rate Alerting & Error Budgets

The one idea: turn an SLO into a budget, then alert on how fast you spend it

An SLO ("99.9% of requests succeed over 30 days") is really a promise about how much failure is allowed. The complement of the target is the error budget: 100% minus 99.9% = 0.1% of events may fail before you have broken the promise. That budget is a fixed quantity per window. The burn rate is how fast you are consuming it, measured relative to the rate that would exactly exhaust it over the whole window. Burn rate 1 means "on track to spend exactly the budget, no more" over the 30 days; burn rate 14.4 means you are spending 14.4× too fast, so the entire monthly budget is gone in about 2 days. One number captures both severity and time-to-exhaustion, which is why you alert on it.

The arithmetic, traced (99.9% / 30 days)

1. Target = 99.9% → budget fraction = 1 minus 0.999 = 0.001 (0.1%).
2. Window = 30 × 24 × 60 = 43,200 minutes.
3. Budget as downtime-equivalent = 0.001 × 43,200 = 43.2 minutes per 30 days (or 0.1% of all requests).
4. At 1,000 req/s the window sees 1,000 × 43,200 × 60 ≈ 2.59 billion requests, so the budget is ≈ 2.59 million allowed errors.
5. Time to exhaust the budget = window ÷ burn rate. So burn 1× → 30 days; burn 14.4× → 30 ÷ 14.4 ≈ 2.08 days; a sustained 5% error rate → burn rate = 0.05 ÷ 0.001 = 50× → 30 ÷ 50 = 0.6 days ≈ 14.4 hours and the whole month's budget is gone.

The link between an error rate and a burn rate is direct: burn rate = observed error rate ÷ budget fraction. A 0.1% error rate on a 99.9% SLO is burn rate 1 (sustainable); 5% errors is 50× (catastrophic). The chart below shows the budget draining at three burn rates.

Multi-window, multi-burn-rate alerts

A single fixed alert forces a bad trade-off. Alert on a high burn rate over a short window and you detect catastrophes fast but page on every brief blip and miss slow leaks. Alert on a low rate over a long window and you catch slow leaks with few false pages but react far too slowly to an outage. Google's SRE Workbook resolves this by running several alerts at once, each pairing a burn rate with a window sized to its severity:

• Fast-burn → page. 14.4× over 1 hour (consumes 2% of the monthly budget in that hour) wakes on-call — a real outage in progress. A second fast tier, 6× over 6 hours (5% of budget), catches slower-but-serious events.
• Slow-burn → ticket. 1× over 3 days (10% of budget) files a ticket, not a page — a quiet leak that would never trip a spike-based alert but will silently eat the month.

Each alert uses two windows: a long one to confirm the burn is significant, and a short one (e.g. 5 minutes alongside the 1-hour) that must also be over rate for the alert to fire. The short window makes the alert reset quickly once the incident ends, so you don't get a stale page lingering for an hour after recovery.

Pitfalls

• Confusing burn rate with error rate. Burn rate is normalized to the budget: the same 1% error rate is burn 10× on a 99.9% SLO but only 1× on a 99% SLO. Always divide by the budget fraction.
• A window with no short partner. A lone 1-hour window keeps paging for up to an hour after the incident is over, because the trailing hour still contains the error spike. The short "and" window is what clears it.
• Alerting without a real SLI. Burn-rate alerts are only as good as the success/failure signal feeding them. A miscounted SLI (e.g. treating 4xx client errors as server failures) makes the budget and every alert wrong.
• Setting the SLO at 100%. A 100% target means zero budget, so any single failure is an infinite burn rate — the whole scheme collapses. SLOs must leave a budget to spend.

Judgment: burn-rate vs the alternatives

Burn-rate alerting vs static threshold alerts. A static threshold ("page if error rate is above 1% for 5 minutes") has no concept of a budget, and it fails three ways. It is noisy: a 2% blip that never threatens the monthly SLO still pages you at 3am. It misses slow burns: a steady 0.5% error rate never crosses a 1% threshold yet quietly exhausts the budget over weeks. And it is not tied to consequence: the same 1% is far too sensitive for a lax SLO and far too lax for a strict one. Burn rate fixes all three by measuring "how fast am I heading toward breaking my promise," so the alert's urgency matches the actual risk to the SLO. The cost of that power is complexity: burn-rate alerting requires a well-defined SLO, a trustworthy SLI, and several coordinated windows — whereas a static threshold is trivial to set up and perfectly adequate for signals that have no SLO at all (disk 90% full, a queue backing up, a cert nearing expiry).

The window/rate tuning trade-off. Detection time is roughly the budget-fraction-to-consume divided by the burn rate, so a high rate over a short window detects a severe outage in minutes but risks false pages on transient spikes and is blind to slow leaks; a low rate over a long window is quiet and catches leaks but is slow to fire. There is no single window that is both fast and quiet — which is exactly why you run multiple: a fast-burn short-window alert for pages and a slow-burn long-window alert for tickets, each with a short partner window to keep it from lingering.

Takeaways

• An SLO implies an error budget (99.9% / 30d ⇒ 0.1% ⇒ 43.2 min); burn rate = error rate ÷ budget fraction, and time to exhaust = window ÷ burn rate.
• Burn rate 1 spends the budget over the full window; 14.4× empties the 30-day budget in ~2 days; a 5% error rate is 50×, gone in ~14.4 hours.
• Run multi-window, multi-burn-rate alerts: fast-burn → page (catch outages in minutes), slow-burn → ticket (catch quiet leaks); a short partner window makes each alert reset fast.
• Prefer burn-rate over static thresholds when you have an SLO — thresholds are noisy and blind to slow burns; keep static thresholds for signals that have no budget.

---

Re-authored from-scratch for this guide. Error-budget and multi-window multi-burn-rate model adapted from the Google SRE Workbook ("Alerting on SLOs") and the SRE Book. Deepened for this guide.

I just read a lesson on **Burn-Rate Alerting & Error Budgets** (System Design) and want to truly understand it. Explain Burn-Rate Alerting & Error Budgets from first principles using ONE vivid real-world analogy and a visual mental model — draw it as ASCII art or a clear step-by-step diagram — with a concrete example using real numbers. Then ask me one question to check I got the mental picture, and wait for my reply. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

Teach me **Burn-Rate Alerting & Error Budgets** interactively. Ask me ONE guiding question at a time, wait for my answer, and adapt to my confusion — build the idea with me step by step instead of explaining it all at once. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

Quiz me on **Burn-Rate Alerting & Error Budgets** with 5 questions, easy to tricky, ONE at a time. Tell me if each answer is right; at the end, explain clearly what I got wrong and why. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

Help me remember **Burn-Rate Alerting & Error Budgets** for the long term: give the one-sentence intuition, a memorable hook/mnemonic, a tiny worked example, and 3 active-recall flashcards (Q -> A). If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.