Stateful vs Stateless Architecture
A stateless server scales horizontally because every request carries (or points to) all the state needed to serve it, so the load balancer is free to send the next request to whichever replica is least busy; a stateful server can't do that — it pins each client to the one node whose RAM holds that client's session, so you can add machines but not rebalance the clients already stuck to a hot one.
That single sentence is the whole scaling story. "Stateless" does not mean the system forgets things — it means the request-handling tier holds no per-client memory. State still exists; it has been moved off the app node into something shared (Redis, a database) or into the client's own credential (a signed token). Where you put that state is the real design decision, and it is the one the bullet-list version of this topic skips.
Worked example: one login, three ways to scale it
Setup: an e-commerce site with 3 app servers (A, B, C) behind a round-robin load balancer. A user logs in, then loads their cart. We trace the same two requests under each strategy. The state we care about is the session sess_abc123 = { user: 42, cart: [] }.
Strategy 1 — in-memory session (naïve stateful)
| Step | Request | LB routes to | What happens |
|---|---|---|---|
| 1 | POST /login | Server B | B validates password, creates sess_abc123 in B's local RAM, returns Set-Cookie: SID=sess_abc123. |
| 2 | GET /cart (cookie SID=sess_abc123) | Server A | A has never heard of sess_abc123 — it lives only in B's memory. A returns an empty cart / 401. User appears logged out. |
Why the naïve version is wrong: round-robin sends request 2 to a different node than request 1, and session memory is node-local. It "works" only when N = 1 server, which is exactly the case you scale out of.
Strategy 2 — sticky sessions (session affinity)
Patch the LB to pin a cookie/IP to one node (hash(SID) → B). Now both requests hit B and the cart works. But you have traded away the thing you were scaling for:
- Under a 5× Black-Friday spike you autoscale 3 → 15 nodes. The 12 new nodes only receive new users; every already-logged-in customer stays pinned to A/B/C. The old three sit at 95% CPU while the new twelve idle — you added capacity you cannot use.
- Deploy or crash Server B and every session on it evaporates: those users are logged out mid-checkout. Scaling in (removing a node) does the same on purpose.
Strategy 3 — externalize state, and the tier goes stateless
| Step | Request | LB routes to | What happens |
|---|---|---|---|
| 1 | POST /login | Server B | B validates, then SET sess_abc123 {user:42,cart:[]} EX 1800 in Redis. Returns the cookie. |
| 2 | GET /cart | Server A | A does GET sess_abc123 from Redis (~0.3 ms), gets {user:42,cart:[]}, serves the cart. Any node can serve any request. |
Now the LB routes freely and all 15 nodes share the load evenly; a dying node loses no session because the session lives in Redis, not in the node. The cost you accepted: one extra network hop per request and a new hard dependency — Redis must now be highly available, because if the shared state tier is down, every node is down.
The auth variant — token-based (JWT), no lookup at all
Identity is a special kind of state: it is small and, within a session, unchanging. So instead of storing it, sign it. At login, Server B mints a JWT with claims { sub: 42, exp: 1751500000 } and an HMAC-SHA256 signature over a secret shared by all nodes. On request 2, any node verifies the signature locally (no Redis, no DB) and trusts sub: 42. This is maximally stateless for authentication — zero shared reads on the hot path. The catch: the cart (mutable, growing) still belongs in Redis/DB. A token is the right home for identity, the wrong home for state that changes during the session.
Pitfalls
- "Stateless" is a relocation, not a deletion. Moving sessions to Redis makes the app tier stateless but makes Redis a hard, shared dependency. If it isn't replicated/failed-over, you converted N independent failure domains into one.
- Sticky sessions quietly cancel load balancing. Affinity keeps a user on one node, so a slow/hot node stays hot and autoscaled nodes stay empty. Combined with rolling deploys or scale-in, it also drops sessions on node termination — users logged out mid-flow.
- JWTs can't be un-issued. A signed token is valid until
exp; "log out" on the client doesn't invalidate it server-side. If a token leaks, it works until expiry. Mitigate with short TTLs + refresh tokens, or a small server-side revocation/denylist — which reintroduces a lookup (a partial return to stateful). - Don't stuff mutable state into the token. Cookies cap around 4 KB and servers reject oversized headers; a fat JWT also goes stale the instant the underlying data changes, because nothing re-reads it. Tokens are for identity and coarse claims, not for the cart.
- Hot keys and thundering herds. A shared session store concentrates traffic; a popular key or a mass token-expiry can spike it. Add TTL jitter and consider local read caches with short TTLs.
- Genuinely stateful protocols resist this. A WebSocket, a gRPC stream, or a game/match server holds live connection state on one node by nature — you can't round-robin its packets. There, per-connection affinity is correct, not a smell.
When to use it / when NOT to
The decision is not "stateful vs stateless" in the abstract — it's where does per-client state live, chosen per workload.
- Token-based (JWT / signed credential) — pick when you need auth/identity at scale, across many services, with zero hot-path lookups; the state is small and stable for a session. Costs: hard revocation, size limits, secret rotation/clock-skew handling. Alternative it beats: server-side session lookup — you gain a network round-trip per request, you lose instant logout.
- Externalized session store (Redis/DB) — pick when you have mutable per-session data (cart, wizard progress) shared across many replicas and you need server-side revocation and freshness. Costs: +0.3–1 ms hop per request, plus you must run the store HA. Alternative it beats: in-memory sessions — you gain free rebalancing and crash-survival, you pay for one more moving part.
- Sticky sessions — pick when it's a quick bridge for legacy in-memory sessions, or the protocol is inherently connection-stateful (WebSocket/streaming/game). Costs: uneven load, sessions lost on deploy/scale-in. Don't reach for it as your scaling strategy for plain HTTP.
Choose stateless app nodes + token for identity + shared store for mutable state when the workload is request/response HTTP that must autoscale and survive node churn (REST APIs, microservices, web front-ends). Prefer a stateful node with affinity when the connection itself is the state — long-lived sockets, real-time streams, in-memory game rooms — where shipping that state off-node every packet would cost more than it saves.
Takeaways
- Stateless scales horizontally because the LB can route any request to any replica — the state isn't on the node.
- You never delete state; you relocate it: into a signed token (identity) or a shared store (mutable session). Pick the home per data type.
- Sticky sessions are a stopgap that trades away the load balancing you were scaling for — and lose sessions on any node churn.
- Tokens buy lookup-free auth but cost you easy revocation; externalized stores buy freshness and revocation but cost a hop and an HA dependency.
High-scale JWT revocation without a database hit on every request
A server-side revocation denylist is the escape hatch for stateless JWTs, but implementing it naively as SELECT revoked WHERE jti = ? on every request destroys the point of local token verification. The production version keeps the revocation check in fast, bounded data structures.
- Redis TTL-based keys. On revocation, write
SET revoked:<jti> 1 EX (exp - now). The key automatically disappears when the token would have expired anyway, so storage is proportional to revoked live tokens, not all issued tokens. - Local memory cache with pub/sub sync. Each API node keeps a small in-memory set of recently revoked
jtis and subscribes to a Redis pub/sub channel. A logout publishes thejti; every node updates local memory within milliseconds. Redis remains the source for recovery after restart, while most requests avoid a network hop. - Local Bloom filter as the negative fast path. A periodically refreshed Bloom filter of revoked IDs lets a node answer "definitely not revoked" locally for almost every token. Only a Bloom-filter hit, which may be a false positive, falls through to Redis for confirmation. This lets the API avoid network hits for non-revoked tokens while preserving correctness.
The trade-off is propagation delay and operational complexity: local caches and Bloom filters are fast, but you must handle node restart, missed pub/sub messages, filter refresh, and false positives. Use this only where instant revocation matters enough to pay that cost.
Re-authored and deepened for this guide. Sources: Roy T. Fielding, Architectural Styles and the Design of Network-based Software Architectures
(2000, Ch. 5 — the REST statelessness constraint); Martin Kleppmann, Designing Data-Intensive Applications
(state, replication, and shared stores); AWS documentation on Elastic Load Balancing session affinity and stateless application tiers; and the DesignGurus Grokking the System Design Interview
treatment of stateful vs stateless services. The decorative comparison GIF was replaced with a hand-authored request-routing diagram.
🤖 Don't fully get this? Learn it with Claude
Stuck on Stateful vs Stateless Architecture? Open Claude, copy a block below, and it'll teach you this exact concept — visually and interactively.
Build the mental picture, not memorization.
I just read a lesson on **Stateful vs Stateless Architecture** (System Design) and want to truly understand it. Explain Stateful vs Stateless Architecture from first principles using ONE vivid real-world analogy and a visual mental model — draw it as ASCII art or a clear step-by-step diagram — with a concrete example using real numbers. Then ask me one question to check I got the mental picture, and wait for my reply. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.
Socratic — adapts to where you're stuck.
Teach me **Stateful vs Stateless Architecture** interactively. Ask me ONE guiding question at a time, wait for my answer, and adapt to my confusion — build the idea with me step by step instead of explaining it all at once. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.
Active recall exposes what you missed.
Quiz me on **Stateful vs Stateless Architecture** with 5 questions, easy to tricky, ONE at a time. Tell me if each answer is right; at the end, explain clearly what I got wrong and why. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.
Intuition + hook + flashcards for long-term memory.
Help me remember **Stateful vs Stateless Architecture** for the long term: give the one-sentence intuition, a memorable hook/mnemonic, a tiny worked example, and 3 active-recall flashcards (Q -> A). If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.