What is a Proxy Server

A proxy works by terminating the client's connection and opening its own, separate connection onward — it never just relays packets, it fully absorbs one connection and re-originates a new one, which is exactly why it can rewrite, cache, filter, or log the traffic in between. A forward proxy sits in front of one or more clients and applies that mechanism outward: it accepts the client's connection, then opens its own connection to whatever the client asked for on the internet, and relays the response back. The origin server never talks to the client directly — it only ever sees the forward proxy.

A forward proxy hides the client's identity from the server: every request the origin sees appears to come from the proxy, not from the individual machine behind it.

Because the proxy is a full man-in-the-middle by design, it is a natural place to add: caching (serve a repeat request without going out again), filtering (block requests to disallowed domains), logging (a single audit point for every client), and request/response transformation (adding or stripping headers, encrypting/decrypting, or compressing a body).

Reverse proxy

A reverse proxy applies the identical mechanism in the other direction: it sits in front of one or more backend servers, terminates the client's connection, and opens its own connection to whichever backend should handle the request, then relays the response back to the client.

Contrary to a forward proxy, which hides the client's identity, a reverse proxy hides the server's identity: the client can see it talked to something, but not which backend, or how many, actually served the request.

Reverse proxies are the usual home for load balancing, TLS termination (so backends never handle raw TLS handshakes), response caching at the edge, and path-based routing to different backend services.

A traced example: collapsed forwarding

Proxies can also optimize traffic system-wide, not just per-request. Collapsed forwarding merges multiple in-flight requests for the same uncached data into a single upstream fetch, then fans the one result out to every waiter. Trace it with real timestamps for three clients requesting the same cache-missed key, GET /user/8842, within a few milliseconds of each other:

Net effect: 3 requests in, 1 disk read out. Without collapsing, all three would have hit the disk independently in the same few-millisecond window.

The failure branch is the interview follow-up: collapsing correlates the waiters' fates — when the single leader fetch fails or times out, the proxy naively fans out N identical errors at the same instant, and the clients retry in lockstep, so the herd is deferred, not killed. Production single-flight therefore (a) caps how many waiters may attach before spilling to load-shedding, (b) staggers waiter retries with jitter so the re-arrival is spread rather than synchronized, and (c) treats a failed leader fetch as a signal to serve stale (stale-if-error) rather than fan out N identical errors.

Pitfalls

• Thundering herd without collapsing: if the proxy does NOT deduplicate in-flight identical reads, a cache-miss burst (e.g. a popular key expiring under load) sends N simultaneous requests straight to the origin/disk — the exact failure collapsed forwarding exists to prevent.
• Forward proxy as a single point of failure: routing all client egress through one forward proxy turns it into both a bottleneck and an outage risk — if it goes down, every client behind it loses internet access, not just one.
• Retry amplification at a reverse proxy: a reverse proxy that blindly retries a slow or failing backend on timeout can multiply load onto an already-struggling server, turning a partial slowdown into a full outage.
• Trusting proxy-set headers blindly: a backend that trusts X-Forwarded-For or X-Real-IP without confirming the request actually came through the trusted proxy can be trivially spoofed by a client that sets those headers itself.
• TLS termination creates a plaintext hop: once a reverse proxy terminates TLS, the connection from proxy to backend is a new one — if that internal hop isn't also encrypted, sensitive data now travels in the clear on the internal network.

When to use which — and the trade-off

Forward proxy when you're protecting or controlling a set of clients: egress filtering, content policy, caching for an internal network, or hiding client identity from the outside world. Reverse proxy when you're protecting or optimizing a set of servers: TLS termination, response caching at the edge, path-based routing, or load balancing across backends.

A common but imprecise claim is "every load balancer is a reverse proxy." That's only true for load balancers that actually terminate the client's connection and open a new one to a backend — an L7 (HTTP-aware) balancer, or an L4 balancer running in full-NAT/proxy mode, both fit this page's own definition of a proxy. It does not hold for packet-forwarding L4 balancers such as LVS/IPVS in direct server return (DSR) mode, or plain NAT-based forwarding: these rewrite the destination address and forward the same packets without ever terminating the client's TCP connection, so by this page's own definition they are not proxies at all, reverse or otherwise. The accurate statement is: many reverse proxies balance load, and many balancing load balancers are reverse proxies — but neither implication is universal in either direction (a reverse proxy can front a single backend with no balancing logic at all, and a DSR/NAT load balancer balances without proxying).

Trade-off vs the named alternative (packet-forwarding L4 / DSR): choose a reverse proxy when you need L7 smarts — header rewriting, cookie-based routing, TLS termination, response caching, request inspection — and can absorb the cost of terminating and re-establishing a connection on every request (extra CPU, extra hop, extra latency, and the proxy itself becomes a scaling/failure point to manage). Choose a packet-forwarding L4 balancer / DSR when you need close-to-line-rate throughput and minimal added latency, don't need to inspect or rewrite payloads, and can accept that the balancer can't see or filter L7 content, can't terminate TLS, and return traffic bypasses it entirely (which is also what makes DSR so fast).

Takeaways

• Both proxy directions are the same mechanism — terminate a connection, open a new one — aimed at hiding a different party: forward hides the client, reverse hides the server.
• Collapsed forwarding turns N concurrent identical cache-misses into 1 upstream read; skipping it is a direct path to a thundering-herd outage.
• "Load balancer" and "reverse proxy" overlap heavily but are not synonyms — a packet-forwarding L4 balancer (e.g. DSR) balances without ever proxying, and a reverse proxy can serve one backend with no balancing at all.
• Whichever you deploy, treat proxy-set headers as untrusted unless you control the network path, and re-secure any hop TLS termination leaves in the clear.

Sources: Grokking the System Design Interview (DesignGurus) — Basics of a Proxy Server; MDN, "Proxy servers and tunneling"; Julia Evans, "Load balancing" notes; LVS/IPVS documentation on Direct Server Return. Re-authored/Deepened for this guide. See also Proxy vs Reverse Proxy vs LB — Traced With a Real Request for a full request trace distinguishing all three.