Networking at Scale — NAT Exhaustion, L4-vs-L7-vs-DSR, QUIC Limits, MTU/PMTUD & CIDR (Deep Dive)

Every layer above IP hides a finite table, and scale bugs are almost always that table running out

A NAT gateway has a finite pool of source ports per destination. An L7 load balancer has a finite pool of worker threads holding TLS state. A connection pool has a finite number of sockets. A CIDR block has a finite number of addresses. None of these are bandwidth problems — they are counting problems: a resource that looks unlimited at low traffic and hard-fails once demand crosses its table size. This page is the practitioner's tour of those tables — where they live, how they fail, and how a senior engineer reasons about which layer to spend cost on. It builds directly on this guide's existing networking pages: The Layered Network Model — OSI, TCP/IP, and L4 vs L7 and IP Addressing, Subnets & CIDR for the basics, TCP Deep — Handshake Cost, Congestion Control & Head-of-Line Blocking and Bandwidth-Delay Product for transport mechanics, TCP Incast for fan-in collapse, and Subnetting & CIDR: usable addresses and the overlap trap for address-block sizing — none of that is re-derived here; it is assumed and extended.

1. NAT and the SNAT port table: why fan-out to ONE destination is the dangerous shape

NAT (network address translation) rewrites a packet's source IP and port as it leaves a private network, and keeps a connection table entry so return traffic can be un-translated back to the right internal host. For outbound (source) NAT specifically, the table's key is effectively the 4-tuple the far side will see: (translated-src-ip, translated-src-port, dst-ip, dst-port). Because the translated source IP is usually fixed (one or a few egress IPs) and the destination is fixed once you pick a downstream service, the only thing that can vary to keep connections distinguishable is the translated source port — and there are only ~64,512 of those (the 16-bit port space minus the ~1,024 reserved low ports). That budget is per (dst-ip, dst-port) pair, and it is shared across every internal host behind that NAT IP, not per-host. This is why the dangerous topology is specifically many source hosts → one popular destination through a shared NAT (many pods calling one database or one third-party API gateway): the port budget doesn't grow with the number of callers, only with the number of egress IPs and destinations.

Traced example: port exhaustion arithmetic

Notice what does not appear in this arithmetic: pod count. Whether it's 200 pods or 20,000 pods behind that NAT, the ceiling is the same 64,512, because the constraint is keyed by destination, not by caller. This is also why the failure is intermittent and confusing in production — most requests succeed (there's almost always a free port most of the time), and only the requests that land when the table is momentarily full fail, which looks like flaky networking rather than a capacity limit.

Mitigations, in the order a senior engineer reaches for them

• More source IPs — each additional egress IP multiplies the port budget for that destination (2 IPs → ~129,024 ports). Cheapest lever when the destination itself can't change.
• Connection pooling / reuse (keep-alive) — shrinks h in the Little's Law sense that matters: instead of one connection per request, a small pool of long-lived connections serves many requests, so the connection arrival rate (not the request rate) is what consumes ports.
• Direct routing / bypass the shared NAT — VPC peering, a dedicated NAT per destination, or giving hot destinations more egress IPs so fan-out traffic isn't all funneled through one shared table.

2. L4 vs L7 vs DSR: where the load balancer sits changes what it costs

The three load-balancing shapes differ in how much of the connection the LB owns, and that difference is exactly what makes each one expensive or cheap in a particular direction.

• L4 (transport passthrough) — the LB forwards packets by IP/port without looking inside them. No TLS termination, no per-request buffering, no application visibility. It's cheap to scale (stateless-ish, mostly just packet forwarding) but it can't route on URL path, header, or content, and it can't do TLS termination/WAF/auth at the LB.
• L7 (application, terminating) — the LB terminates the client's TLS connection, parses the HTTP request, and opens a second connection to the backend (re-encrypting if needed). This buys content-based routing, centralized TLS/cert management, WAF, and per-request observability — at the cost of being a genuinely stateful, CPU/memory-bound tier: every open connection holds TLS session state and request/response buffers, and every new TLS handshake costs real CPU (asymmetric crypto ops). At high connection counts or connection churn, the L7 tier itself becomes the bottleneck — not the network, not the backend.
• DSR (direct server return) — the LB forwards the inbound request, but the backend replies directly to the client, bypassing the LB entirely on the return path (the backend holds a loopback alias of the LB's virtual IP so the client sees a consistent source). This is the standard trick for asymmetric traffic — small requests, large responses (video, large file downloads, streaming) — because the LB only ever forwards the cheap direction; the expensive direction (bulk response bytes) never touches it. The cost is topology rigidity: DSR generally needs the LB and backends on the same L2 segment (or a tunnel), and it can't easily do L7-style content inspection since it never sees the reply.

Cost ordering and when each wins

Cost to operate climbs L4 → L7 (stateless forwarding → stateful termination), while DSR trades LB cost for topology constraints rather than removing cost outright. A senior engineer picks based on what the traffic shape demands: symmetric small request/response and cost-sensitive fan-out → L4; need content-aware routing or centralized TLS/WAF → L7 (and budget for its CPU/memory); wildly asymmetric payloads (video, big downloads) where the LB would otherwise choke on return traffic → DSR.

3. QUIC's limits — it is not "TCP but strictly better"

QUIC's headline pitch is fixing TCP's cross-stream head-of-line (HOL) blocking: over TCP, HTTP/2 multiplexes many logical streams onto one ordered byte stream, so one lost packet stalls every stream until it's retransmitted, even streams that had nothing to do with the loss. QUIC gives each stream its own independent loss-recovery sequence, so a lost packet on stream A no longer blocks stream B. That is a real and valuable fix — but it is narrower than it sounds, and it comes with real costs the guide should name explicitly:

• HOL blocking still exists WITHIN a stream. QUIC removes cross-stream blocking, not intra-stream ordering — a lost packet on stream A still stalls stream A's own delivery until it's recovered, exactly like TCP would for a single connection. If your workload is dominated by one big stream (a single large download), QUIC gives you nothing over TCP for HOL blocking.
• QPACK can reintroduce ordering constraints. HTTP/3's header compression (QPACK, the QUIC analogue of HTTP/2's HPACK) uses a dynamic table shared across streams. If a packet carrying a table update is lost, decoding headers on other streams that reference that table entry can stall until it arrives — a narrower, but real, reintroduction of cross-stream dependency.
• 0-RTT is replayable. QUIC's 0-RTT mode lets a returning client send application data in its very first flight, before the handshake completes — great for latency, but those early packets have no protection against replay (an attacker can capture and resend them). It is only safe for idempotent requests (e.g. GET); a non-idempotent request (a payment, a "transfer funds" POST) must not be sent as 0-RTT data.
• Userspace cost. QUIC runs over UDP in userspace, so packet processing (parsing, congestion control, ack handling) that the kernel does for free for TCP now costs CPU in the application/library — at high packet rates this is a measurable CPU tax versus kernel-offloaded TCP.

When QUIC helps vs. hurts

QUIC earns its keep on lossy or variable networks (mobile, Wi-Fi, satellite) and workloads with genuinely independent multiplexed streams (many small requests over one connection, connection migration across a network change) — exactly the case where cross-stream HOL blocking on TCP hurts most. It is the wrong tool on low-loss, CPU-constrained, high-throughput paths — intra-datacenter service mesh traffic, for instance, where kernel TCP with hardware offload is cheaper per byte and loss is rare enough that HOL blocking rarely triggers anyway.

4. DNS is part of the connection, not a step before it

It's easy to reason about connection cost as "TCP handshake + TLS handshake" and forget the RTT that happens before either: resolving the hostname. A cold connection's true latency is:

So a fully cold request over TLS 1.3 costs roughly 3–4 RTTs before the app even sees the response — and DNS is a full quarter to a third of that, invisible in most latency dashboards because it's not "the connection," it's what happens before it. This matters most for tail latency: any request that happens to hit a DNS cache miss, a cold TCP+TLS pair, and a busy backend stacks all of those RTTs, which is why the p99 can be several multiples of the median. It's also why connection reuse (keep-alive pools) and pre-connect (warming a DNS+TCP+TLS-established connection before it's needed, e.g. browsers' rel=preconnect or a backend pre-warming its downstream pool) are disproportionately effective latency fixes — they eliminate 3–4 RTTs of setup cost entirely, rather than shaving milliseconds off the request itself.

5. MTU, MSS, and the PMTUD blackhole — "small payloads work, large ones hang"

Every link on a path has a maximum transmission unit (MTU) — the biggest frame it can carry. TCP negotiates a maximum segment size (MSS) at handshake time assuming the standard Ethernet path MTU (1,500 bytes → MSS ≈ 1,460), and relies on Path MTU Discovery (PMTUD) to shrink that assumption if some hop on the path has a smaller MTU. PMTUD works by setting the "Don't Fragment" (DF) bit on outgoing packets; if a router can't forward a DF packet because it's too big for the next hop, it drops the packet and sends back an ICMP "Fragmentation Needed" message telling the sender to use a smaller size. The mechanism completely depends on that ICMP message getting back to the sender — and a very common piece of network hardening is to block inbound ICMP at a firewall, which silently breaks PMTUD without anyone intending to.

Traced example: the blackhole

The diagnostic fingerprint to remember: small requests succeed, large ones hang or time out, and it "worked yesterday on a different network path." That combination points straight at a PMTUD blackhole before you look at anything else.

6. Long-lived streams (WebSocket/gRPC) pin a connection to one worker for its whole life

A WebSocket connection or a gRPC stream is not a short request/response — it's a connection meant to stay open for minutes or hours. Wherever an L7 tier sits in the path, that tier must keep the stream pinned to one specific backend worker for the stream's entire lifetime (it can't hand the stream off mid-flight the way it can route a fresh short request to any healthy backend). Two consequences follow directly:

• Worker exhaustion. Each long-lived stream permanently occupies one worker slot on the L7 tier for as long as it's open. A burst of long-lived connections (many users opening a live dashboard, a chat app reconnect storm) can exhaust the L7 tier's worker capacity even though total request/response throughput looks modest — the bottleneck is concurrent held connections, not bytes moved.
• Slow connection draining. Rolling a deployment normally drains a backend by waiting for its in-flight short requests to finish, which takes seconds. A backend holding open WebSocket/gRPC streams can't drain that fast — the operator must wait for those streams to end naturally or forcibly close them (breaking active sessions), which is why long-lived-stream services need explicit drain timeouts and client-side reconnect logic.

Where each tier fits: an L4 tier carries long-lived streams cheaply — it's just forwarding packets for the connection's life, no per-stream buffering cost. An L7 tier can carry them but pays the worker-pinning cost above. DSR is a poor fit for bidirectional long-lived streams (it's built around a request-in/response-out asymmetry, not a persistent two-way channel).

7. Connection-pool downsides, and CIDR at scale

Connection pooling (§1's mitigation for NAT exhaustion) is not free. Idle pooled connections still hold a socket, file descriptor, and often TLS session state on both ends — a large pool sitting mostly idle is memory and FD pressure on the server even when it's doing no work. Worse, a pooled connection can silently point at a backend that's since been drained or rotated out: unless the pool actively health-checks idle members, a client can keep sending requests down a "connection" whose far end is gone until a request actually fails and forces re-validation. Separately, many low-latency systems disable Nagle's algorithm (TCP_NODELAY) to avoid the ~40 ms coalescing delay it can add to small writes — but that trade increases the raw packet-per-second rate for the same payload, and PPS (not bandwidth) is what saturates many NICs and middleboxes first.

CIDR block sizing has the same finite-table character at cluster scale. This guide's Subnetting & CIDR: usable addresses and the overlap trap already covers the core arithmetic (every subnet reserves network + broadcast addresses, and cloud providers reserve more on top, so sizing off the "total" address count over-provisions on paper and under-provisions in practice) and the VPC-peering overlap trap — that material is assumed here. Two scale-specific failure modes worth adding on top:

• Per-pod CIDR exhaustion in Kubernetes. Each node is typically allocated a slice (e.g. a /24) out of the cluster's overall pod CIDR to hand out one IP per pod on that node. If the cluster CIDR is sized for "how many nodes we have today" rather than "how many we'll ever scale to," adding nodes eventually fails — not because compute capacity is missing, but because there's no more pod-address range left to allocate to a new node. This is a pure address-table exhaustion, identical in shape to SNAT port exhaustion in §1, just at a different layer.
• CIDR overlap breaks peering, silently until it doesn't. Two VPCs (or a VPC and an on-prem network over VPN) that were provisioned independently can easily end up with overlapping address ranges (both picked 10.0.0.0/16, say). Peering or a VPN between them then can't unambiguously route — traffic meant for 10.0.5.4 in one network is indistinguishable from 10.0.5.4 in the other. This is invisible until the day someone tries to connect the two networks, at which point the fix is a renumbering project, not a config change — which is why address planning has to happen before networks are stood up, not after.

Pitfalls

• Blaming "the network" for NAT exhaustion. Symptoms (intermittent connection refused/timeout under load, fine at low traffic) look like generic flakiness; the actual cause is a specific (dst-ip, dst-port) port table filling up, and it's fixed by pooling/more egress IPs, not by "adding more bandwidth."
• Assuming QUIC removes all head-of-line blocking. It removes only the cross-stream case; a workload dominated by one big stream, or with a shared QPACK dynamic table, still stalls.
• Sending non-idempotent requests as QUIC 0-RTT. Replayable by construction — a payment or state-mutating call sent as 0-RTT can execute twice if an attacker replays the captured packet.
• Ignoring DNS in latency budgets. Teams tune TCP/TLS handshake cost and never notice that a DNS cache miss is adding a full extra RTT to the cold-connection path, especially at p99.
• Treating "large payload hangs, small payload works" as an application bug. It is very often a PMTUD blackhole from ICMP-blocking middleboxes — check MSS clamping before debugging the app.
• Choosing DSR without checking topology. DSR needs the backend to own the LB's VIP on the return path (same L2 segment or a tunnel); bolting it onto an L3-routed, NAT'd topology doesn't work without extra tunneling machinery.
• Sizing a cluster's pod CIDR off current node count. Under-sizing the pod CIDR is a silent ceiling on cluster growth that isn't visible until nodes start failing to join.

Judgment layer: how a senior engineer decides

L4 vs L7 vs DSR

TCP vs QUIC

Takeaways

• NAT port tables, L7 worker pools, connection pools, and CIDR blocks are all the same shape of problem: a finite table whose capacity should be checked against demand using Little's Law (concurrent = arrival rate × hold time) — not against raw bandwidth.
• A cold connection costs DNS + TCP + TLS RTTs stacked together, not just "the handshake" — connection reuse and pre-connect are bigger latency levers than swapping transport protocols.
• QUIC fixes cross-stream head-of-line blocking on lossy/mobile networks; it does not remove intra-stream blocking, does not make 0-RTT safe for non-idempotent requests, and costs more CPU per packet than kernel TCP.
• "Small payloads succeed, large payloads hang" is the fingerprint of a PMTUD blackhole caused by ICMP-blocking middleboxes — fix it with MSS clamping at the constrained hop, not application-level retries.

Related pages

• Load Balancing — Tier Sizing, Retry Storms, Load-Shedding, GSLB & L7-vs-Volumetric (Deep Dive) — the load-balancer capacity and tier-sizing math behind this page's L4-vs-L7-vs-DSR cost trade-offs.
• Scalable Systems Advanced — Workload Identity, QUIC Under Loss, Late-Data Handling & Burn-Rate Alerting (Deep Dive) — a deeper look at QUIC's behavior under loss, complementing this page's QUIC-limits section.
• Difference Between LongPolling, WebSockets, and ServerSent Events — the transport trade-offs behind this page's long-lived-stream worker-pinning discussion.
• What Is the Difference between Liveness Checks and Readiness Checks in Load Balancers — relevant to the slow-drain problem this page describes for long-lived streams during rolling deploys.
• Cache Stampede & Invalidation — Thundering Herd and the Hard Half — the same finite-table/thundering-herd shape as this page's NAT/CIDR exhaustion, applied to caches.