Uses of Load Balancing
Load balancing distributes incoming work across multiple computing resources so that no single resource is overwhelmed. The benefit is not automatic: the load balancer must first terminate or observe the traffic, then pick a healthy backend using some rule (round-robin, least-connections, hash, etc.), and finally forward the request. Each use below is the visible result of that mechanism, and each pays a cost — an extra hop, operational complexity, or a new failure domain — that should be weighed against the gain.
1. Improving website performance
How it works: By fanning requests across N identical backends, the queue of waiting requests at each server shrinks. If 10,000 QPS hits one server, a long queue builds and tail latency explodes; split across 5 servers and each one sees 2,000 QPS, so requests wait far less.
What it costs: You now pay one extra network hop (roughly 1–3 ms for an L7 load balancer in the same AZ) and you must size the LB itself so it does not become the bottleneck.
Worked example (M/M/1 approximation — every number derived). One server processes a request in 9 ms of work and receives 100 QPS → utilization ρ = 100 × 0.009 = 0.9. In an M/M/1 approximation the mean response time is W = S/(1−ρ) = 9/(1−0.9) = 90 ms — ten times the service time, because queueing delay grows as 1/(1−ρ) and explodes near saturation. Put 5 identical servers behind a load balancer: each sees 20 QPS, ρ = 0.18, W = 9/0.82 ≈ 11 ms. Throughput per server dropped 5×, but latency dropped ~8× — the win is nonlinear, because you moved off the hockey-stick part of the 1/(1−ρ) curve. That curve, not raw QPS division, is why load balancing improves performance. The hop through the LB adds ~1 ms, so the net latency win is dominated by escaping the queueing regime.
2. Ensuring high availability and reliability
How it works: The load balancer runs periodic health checks against each backend. When a backend fails its probe, the LB stops sending traffic to it; traffic is rerouted to the surviving healthy nodes.
What it costs: The load balancer itself becomes a potential single point of failure. You must run at least two LBs behind a floating VIP (VRRP/keepalived, anycast, or DNS) and tune probe intervals so they are fast enough to detect failure but not so aggressive that a GC pause ejects a healthy node.
Example: A banking application relies on several servers to handle user transactions. The load balancer monitors the health of each server and, in the event of a server failure, redirects traffic to the remaining healthy servers, minimizing downtime and maintaining user access to the application.
3. Scalability
How it works: New backends are registered in the LB's pool, and the balancer immediately starts sending a share of traffic to them. The scaling action is horizontal — add more small boxes rather than buying one bigger box.
What it costs: Scaling only works for stateless or shared-state workloads. If a backend holds in-memory session state, adding backends can break logged-in users unless sessions are moved to Redis or another shared store.
Example: A video streaming platform sees a steady increase in users as it gains popularity. To handle the growing demand, the platform adds new servers to the load balancing pool, allowing it to scale seamlessly without overloading existing infrastructure.
4. Redundancy
How it works: Multiple copies of a service are kept live behind the LB. If one copy fails, the others continue serving traffic while the failed copy is repaired or replaced.
What it costs: Redundancy is not the same as durability. A load balancer does not create backups of your data; it only routes around failed compute. Data redundancy still requires replication, backups, or object storage.
Example: An online file-storage service runs its API tier as five identical stateless servers behind a load balancer, while the files themselves are replicated by the storage layer (three copies in the object store). When one API server dies of a hardware fault, the health check ejects it and requests flow to the surviving four — availability of the service is preserved by the load balancer; durability of the data is preserved by storage replication, a separate mechanism the LB knows nothing about.
5. Network optimization
How it works: Traffic is spread across multiple network paths or uplinks so that no single link saturates.
What it costs: Equal-cost multi-path (ECMP) or link aggregation gives throughput but not always perfect balancing; a single long flow may still pin to one path, and asymmetric routing can complicate debugging.
Example: A large organization has multiple internet connections to handle its network traffic. A load balancer distributes the incoming and outgoing traffic across these connections, reducing congestion and improving overall network performance.
6. Geographic distribution
How it works: Global Server Load Balancing (GSLB) or anycast steers a user to the nearest healthy region before the regional load balancer picks an instance.
What it costs: DNS-based steering is bound by TTL caching; a client that cached a dying region will keep sending traffic there until the TTL expires. Anycast converges faster but requires BGP control.
Example: A multinational company has data centers in North America, Europe, and Asia. A load balancer directs users to the nearest data center based on their geographic location, reducing latency and improving the user experience.
7. Application performance isolation
How it works: Different applications or tenants can be routed to dedicated backend pools so that a spike in one workload does not starve another.
What it costs: You must maintain separate pools and possibly separate LB rules per application, increasing operational surface.
Example: An enterprise uses a suite of applications, including email, file storage, and collaboration tools. A load balancer assigns dedicated resources to each application, ensuring that each service performs optimally without affecting the performance of other applications.
8. DDoS and abuse resilience
How it works: Distributing traffic across many backends prevents any single server from being the sole target of a flood. Layer-7 LBs can also enforce rate limits and connection caps.
What it costs: A Layer-7 load balancer is itself a volumetric victim: it must terminate TCP and TLS for every connection, so a SYN flood can exhaust it before the traffic even looks like HTTP. Real DDoS defense sits upstream (anycast, scrubbing centers, SYN cookies at L3/L4).
Example: A news website faces a distributed denial-of-service (DDoS) attack, with a large number of malicious requests targeting its servers. The load balancer distributes the traffic among multiple servers, making it more difficult for the attackers to overwhelm a single target and mitigating the impact of the attack.
9. Cost efficiency
How it works: Right-sizing many commodity instances behind a load balancer is often cheaper than one oversized machine, and idle capacity can be removed during low traffic.
What it costs: The savings are real only if the workload scales horizontally. A database-bound or single-threaded application will not become cheaper just because a load balancer sits in front of it.
Example: A small business utilizes cloud-based infrastructure for its web applications. By using load balancing to optimize resource usage, the business can minimize the number of servers needed, resulting in lower infrastructure and energy costs.
10. Content caching
How it works: Some L7 load balancers can cache static content (images, CSS, JS) and serve it directly, avoiding a round-trip to the origin.
What it costs: An LB cache is small and single-region; it is not a substitute for a CDN or a real cache tier. Caching dynamic or personalized content can leak one user's data to another.
Example: A news site's article pages reference the same 200 KB of CSS/JS bundles and a handful of shared images on every request. The L7 load balancer caches those static assets and serves them directly, so during a traffic spike the origin servers spend their cycles rendering articles instead of re-serving identical bytes — while the site's videos and images at global scale stay on the CDN, which is the right tier for large, geographically-spread content (see the CDN lesson).
When NOT to use a load balancer
A load balancer is not free. Skip it when:
- You have a single instance with headroom. The LB adds an extra hop, operational surface, and a new failure domain for zero benefit.
- The bottleneck is not the web tier. If your application is database-bound, adding more app servers behind a LB will not reduce query latency.
- You only need coarse geographic steering. DNS round-robin or anycast may be enough and avoids the operational complexity of a full LB tier.
- You control every client. A service mesh with client-side load balancing can avoid the central hop entirely.
Alternatives at a glance
| Approach | What it does | Best for | Main limitation |
|---|---|---|---|
| Dedicated load balancer | Terminates or observes traffic, picks a healthy backend | Internet-facing, multi-backend, health-aware routing | Extra hop, SPOF unless redundant, operational complexity |
| DNS round-robin | Returns multiple A records; clients pick one | Coarse geo-steering, zero operational cost | No health awareness, TTL caching keeps sending traffic to dead nodes |
| Client-side / service mesh | Each client picks its own backend | Internal service-to-service calls | Every client needs routing logic and health awareness |
| Vertical scaling | Run one bigger server | Simple workloads, low traffic, stateful single-node apps | Hard ceiling, single point of failure |
Interview traps
- "Is a load balancer the same as a reverse proxy?" Every load balancer is a reverse proxy, but not every reverse proxy load-balances. A reverse proxy may forward to a single backend; a load balancer adds the routing decision and health checking.
- "When can a load balancer become the bottleneck?" At TLS handshake rates (CPU), NIC line rate, or connection memory — whichever hits first. You size the LB tier against all three ceilings, not just average QPS.
- "Would you put a load balancer in front of a single database?" Not to scale writes. A database is usually bottlenecked by storage, locks, or replication lag; a load balancer does not fix that. Use it for read-replica routing if the DB layer supports it.
Ten uses, four mechanisms
The ten uses above are not ten separate facts. Each one falls out of one of four underlying verbs the load balancer performs: spread a queue, health-check and reroute, terminate & offload, or place/steer traffic.
| Use | Queue-spreading | Health-aware rerouting | Terminate & offload | Placement/steering |
|---|---|---|---|---|
| 1. Website performance | ✓ | |||
| 2. High availability | ✓ | |||
| 3. Scalability | ✓ | |||
| 4. Redundancy | ✓ | |||
| 5. Network optimization | ✓ | |||
| 6. Geographic distribution | ✓ | |||
| 7. Performance isolation | ✓ | |||
| 8. DDoS resilience | ✓ | |||
| 9. Cost efficiency | ✓ | |||
| 10. Content caching | ✓ |
If an interviewer asks for an eleventh use, derive it from a mechanism rather than reciting this list — anything an LB does is one of these four verbs.
Key takeaways
- Each LB benefit comes from a concrete mechanism — usually spreading a queue, health-checking backends, or caching at the edge.
- Every benefit pays a cost: extra hop, new failure domain, operational complexity, or a ceiling that must be sized.
- Do not use a load balancer reflexively. If one server fits, if the bottleneck is the database, or if you only need coarse DNS steering, skip it.
🤖 Don't fully get this? Learn it with Claude
Stuck on Uses of Load Balancing? Open Claude, copy a block below, and it'll teach you this exact concept — visually and interactively.
Build the mental picture, not memorization.
I just read a lesson on **Uses of Load Balancing** (System Design) and want to truly understand it. Explain Uses of Load Balancing from first principles using ONE vivid real-world analogy and a visual mental model — draw it as ASCII art or a clear step-by-step diagram — with a concrete example using real numbers. Then ask me one question to check I got the mental picture, and wait for my reply. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.
Socratic — adapts to where you're stuck.
Teach me **Uses of Load Balancing** interactively. Ask me ONE guiding question at a time, wait for my answer, and adapt to my confusion — build the idea with me step by step instead of explaining it all at once. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.
Active recall exposes what you missed.
Quiz me on **Uses of Load Balancing** with 5 questions, easy to tricky, ONE at a time. Tell me if each answer is right; at the end, explain clearly what I got wrong and why. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.
Intuition + hook + flashcards for long-term memory.
Help me remember **Uses of Load Balancing** for the long term: give the one-sentence intuition, a memorable hook/mnemonic, a tiny worked example, and 3 active-recall flashcards (Q -> A). If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.