Load Balancing Algorithms — Traced (RR, Least-Conn, Power-of-Two)
"Put a load balancer in front" — but which algorithm?
A load balancer spreads requests across backends, but how it picks the backend decides whether a slow or hot server quietly melts down. The algorithm is the interesting design choice.
The algorithms, and when to pick each
| Algorithm | Idea | Best for / caveat |
|---|---|---|
| Round-robin | next server in rotation | uniform stateless backends; ignores actual load |
| Weighted RR | bigger servers get more | heterogeneous capacity |
| Least-connections | fewest in-flight requests | variable request cost; adapts to a slow backend |
| Least-response-time | fastest observed latency | latency-sensitive; needs live metrics |
| IP / consistent hash | same client → same server | session/cache affinity; minimal reshuffle on scale (see Consistent Hashing) |
| Power-of-two-choices | pick 2 at random, send to the lighter | near-optimal balance at almost the cost of random — the modern default |
The subtle win — power-of-two-choices: pure random can pile up; checking all servers is expensive and causes herding (everyone picks the same "least loaded" one). Sampling just two and taking the lighter gets you exponentially better balance for O(1) work — which is why it underpins many modern proxies.
Recall the L4 vs L7 distinction (Networking Fundamentals): an L4 LB balances by connection (fast, blind to content); an L7 LB reads the request (path/headers) to route — needed for least-response-time and content routing.
Pitfalls
- Round-robin + uneven request cost = the overload in the diagram. Use least-connections.
- Sticky sessions defeat balancing and break on scale-down — prefer stateless backends + shared session store.
- Always pair with health checks (eject failing backends) and the heartbeat mechanism.
Takeaways
- RR/weighted = simple but load-blind; least-connections/response-time adapt to real load.
- Power-of-two-choices = near-optimal balance, O(1) — reach for it by default.
- Consistent hash for affinity; L4 (fast/blind) vs L7 (content-aware) routing.
Re-authored for this guide; load-distribution diagram hand-authored as SVG. Follows Hello Interview, the "power of two choices" result, and Nginx/Envoy docs. See also: Networking Fundamentals (L4/L7), Consistent Hashing, Heartbeat.
🤖 Don't fully get this? Learn it with Claude
Stuck on Load Balancing Algorithms — Traced (RR, Least-Conn, Power-of-Two)? Open Claude, copy a block below, and it'll teach you this exact concept — visually and interactively.
Build the mental picture, not memorization.
I just read a lesson on **Load Balancing Algorithms — Traced (RR, Least-Conn, Power-of-Two)** (System Design) and want to truly understand it. Explain Load Balancing Algorithms — Traced (RR, Least-Conn, Power-of-Two) from first principles using ONE vivid real-world analogy and a visual mental model — draw it as ASCII art or a clear step-by-step diagram — with a concrete example using real numbers. Then ask me one question to check I got the mental picture, and wait for my reply. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.
Socratic — adapts to where you're stuck.
Teach me **Load Balancing Algorithms — Traced (RR, Least-Conn, Power-of-Two)** interactively. Ask me ONE guiding question at a time, wait for my answer, and adapt to my confusion — build the idea with me step by step instead of explaining it all at once. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.
Active recall exposes what you missed.
Quiz me on **Load Balancing Algorithms — Traced (RR, Least-Conn, Power-of-Two)** with 5 questions, easy to tricky, ONE at a time. Tell me if each answer is right; at the end, explain clearly what I got wrong and why. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.
Intuition + hook + flashcards for long-term memory.
Help me remember **Load Balancing Algorithms — Traced (RR, Least-Conn, Power-of-Two)** for the long term: give the one-sentence intuition, a memorable hook/mnemonic, a tiny worked example, and 3 active-recall flashcards (Q -> A). If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.