What Is the Difference between Rate Limiting and Throttling and Quotas

Under the hood all three are the same move: before a request is served, a counter tied to the caller is checked against a limit, and the request either decrements it or is refused. What separates them is only two knobs — how long the window is, and what happens on overflow. Rate limiting uses a short window (seconds) and rejects the overflow; throttling reacts to load by delaying or queuing it instead of dropping it; a quota uses a long window (a day, a billing month) and blocks until the period resets. Everything else on this page is a consequence of those two knobs. So the real content is the counter: the algorithm that decides, on every single request, whether a token is available.

The mechanism: a token bucket

The dominant rate-limiter algorithm (used by AWS API Gateway, Stripe, Envoy; NGINX's limit_req uses the closely related leaky bucket) is the token bucket. Picture a bucket with three parameters:

• Capacity C — the most tokens the bucket can hold. This is your maximum burst size.
• Refill rate r — tokens added per second. This is your sustained rate.
• Cost — tokens a request consumes (usually 1).

The bucket refills continuously toward C. Each request that arrives tries to remove a token: if at least one is present the request is served and a token is spent; if the bucket is empty the request is refused (429). Two numbers per client — current token count and the timestamp of the last refill — capture the entire state. The key insight: a client that has been quiet accumulates a full bucket and can fire C requests instantly (a burst), then is throttled down to the steady rate r once the bucket drains. That is exactly the "short bursts above the steady rate" behaviour the naive counter cannot express.

The whole algorithm, lazily refilled on each request (no background timer needed):

# Per client: state = {tokens: float, last: timestamp}
# CAPACITY = 10 tokens, REFILL_RATE = 5 tokens/sec

def allow(client, now):
    b = state[client]
    elapsed = now - b.last
    b.tokens = min(CAPACITY, b.tokens + elapsed * REFILL_RATE)  # accrue
    b.last = now
    if b.tokens >= 1:
        b.tokens -= 1
        return True     # 200 OK  (or, if throttling: delay then serve)
    return False        # 429 Too Many Requests

Note the refill is computed from elapsed time, not ticked by a clock — that is what keeps it O(1) memory and lock-light per client.

Worked trace: C = 10, r = 5 tokens/sec

The bucket starts full (10 tokens). Watch a burst drain it, watch requests get refused, and watch tokens re-accrue at 5/sec. Each request costs 1 token.

In the first 1.1 seconds the client got 15 requests served. A dumb "5 per second" rule would have allowed roughly 5–6. The extra 9 came out of the pre-filled bucket — bounded burst, then hard convergence to the sustained 5/sec. That single behaviour is why token bucket won.

How the three differ, concretely

The distinction is not the algorithm — a token bucket can power all three — it is window length and overflow policy.

Terminology honesty: vendors do not use these labels consistently — Microsoft calls Graph's 429-with-Retry-After “throttling” even though by this page's taxonomy that is rejection-style rate limiting (the delay is pushed to the client via Retry-After rather than queued server-side). In an interview, define your terms by the two knobs (window, overflow action) and the ambiguity disappears.

They compose: a real gateway runs a per-second token bucket (rate limit), spills the overflow into a short queue on a spike (throttle), and separately tallies a monthly counter per API key (quota). GitHub's 60-req/hour unauthenticated limit is a real, correct example of the rate-limiting knob.

Pitfalls

• The fixed-window boundary burst (why the naive counter is wrong). A tempting implementation is "count requests, reset the counter every 60s." With a 100/min limit a client sends 100 at 12:00:59 and 100 more at 12:01:00 — 200 requests in one second, double the intended rate, because the two bursts land in adjacent windows. Token bucket and sliding-window algorithms don't have this seam; fixed-window counters do. Fix it by using a sliding window or a token bucket.
• Throttling with an unbounded queue. "Don't drop — just delay" quietly turns into a memory leak and latency blow-up: under sustained overload the queue grows without bound, tail latency explodes, and you fail slower instead of faster. Bound the queue and shed load past it.
• Limiting on the wrong key. Rate-limiting by client IP punishes everyone behind a corporate NAT or mobile carrier as if they were one abuser, while a botnet with thousands of IPs sails through. Limit by API key / user / token when you can.
• Per-node buckets in a fleet. Each server keeping its own in-memory bucket means the real limit is N× your intended one behind a load balancer. Global limits need a shared store (Redis) or a coordinated token-distribution scheme — at the cost of a network hop per request.
• Silent 429s with no guidance. Reject without a Retry-After header or remaining/reset headers and well-behaved clients hammer you in a tight retry loop, amplifying the overload. Always tell the client when to come back, and have them add jitter.
• Non-durable quota counters. Quotas span a billing month; if the counter lives only in memory a reboot resets it and users get free calls (or, worse, you double-bill). Quotas need persistent storage; rate-limit counters can be ephemeral.

When to use which — and which algorithm

Two decisions: which overflow policy, and which counting algorithm.

Reject vs. delay vs. block

• Reject (rate limit / 429) when the caller can retry and you want back-pressure pushed to the client — interactive public APIs. Cheapest for the server (no state held). Choose this by default.
• Delay (throttle) when dropping is worse than waiting and you partly control the client — async/batch pipelines, protecting a fragile downstream during a flash sale. You gain graceful degradation; it costs server memory (queues), added tail latency, and queue-management complexity. Prefer rejection when clients are untrusted or numerous.
• Block over a long window (quota) for business/billing enforcement, not real-time protection. It costs durable storage and can't defend against a one-second spike — always pair it with a rate limit.

Counting algorithm

• Token bucket — choose when you want to permit bounded bursts and a smooth sustained rate; O(1) state (two numbers) per client. Trade-off: it does not strictly cap requests in an arbitrary window (a full bucket plus refill can exceed a naive per-window count). This is the right default for most APIs.
• Leaky bucket — choose when a downstream needs a strictly constant output rate (e.g., a queue drained at fixed speed): it converts bursts into steady drip. Costs added latency and disallows the burst that token bucket permits. Prefer token bucket when clients legitimately spike.
• Fixed-window counter — choose only for cheap, rough limits where precision doesn't matter; O(1) and trivial. Costs the 2× boundary-burst error above. Prefer token bucket or sliding window when the limit must be honest.
• Sliding-window log — choose when you need an exact count over any trailing window (strict compliance/security). Costs O(N) memory: one timestamp per request. The sliding-window counter approximates it at O(1) and is usually the better trade.

Rule of thumb: reach for a token bucket rate limiter as the default; add a durable quota for monetization; add throttling only for the specific downstream you must protect from spikes; and never ship a fixed-window counter where the boundary burst would actually hurt.

Takeaways

• All three are a counter checked before serving; they differ only in window length and overflow policy (reject / delay / block).
• The token bucket's two parameters map directly to intent: capacity = max burst, refill rate = sustained rate. Empty bucket → refuse.
• A naive fixed-window counter allows 2× the limit at window boundaries — a real bug; token bucket and sliding windows avoid it.
• Rate-limit for fairness (default: reject), throttle to survive spikes (delay a specific downstream), quota for billing (durable, long-window) — and compose all three at the gateway.

Re-authored and deepened for this guide. Sources: the token-bucket and leaky-bucket algorithms as documented in the AWS API Gateway usage-plan / throttle-and-burst model and NGINX/Envoy rate-limiting docs; the fixed-window boundary-burst problem and sliding-window counter as described in Alex Xu, System Design Interview (Ch. 4, "Design a Rate Limiter"); GitHub REST API rate-limit documentation (60 req/hr unauthenticated); Microsoft Graph throttling guidance (429/503 with Retry-After); and DesignGurus "What is rate limiting?". Worked trace values are illustrative and hand-verified.