CDN Usage vs Direct Server Serving

A CDN puts a physical copy of your bytes in a rack near the user and steers the request there — so the round-trip that dominates web latency shrinks from a cross-ocean 80 ms to a metro-local 5 ms — while direct serving forces every user to reach one origin no matter where they are. The whole trade-off turns on two mechanisms: how a request finds the nearest edge, and what happens when that edge does or does not already hold the object.

How a request reaches the nearest edge

Two mechanisms dominate, and real CDNs use one or both:

• Anycast (Cloudflare, Fastly) — every edge PoP advertises the same IP prefix via BGP. The internet's routing fabric naturally hands each client's packets to the topologically closest PoP. One DNS answer, no geo-logic in DNS; the network does the steering.
• DNS steering / unicast (classic Akamai) — your hostname is a CNAME to the CDN. The CDN's authoritative DNS looks at the resolver's location (or EDNS Client Subnet) and returns the IP of a nearby PoP. Steering happens at resolve time, so it inherits DNS TTL staleness.

Once at the edge, the request keys into the edge cache (URL + selected headers, e.g. Vary: Accept-Encoding). A hit is served from the PoP's RAM/SSD in single-digit ms. A miss triggers an origin pull: the edge fetches from origin over an already-warm, pooled TLS connection, stores the object per its Cache-Control, and streams it back. So the first user in a region pays the origin round-trip; everyone after them rides the cache.

Worked example: one image, London user, origin in us-east-1

Origin is in Virginia; the London↔Virginia network RTT is ~80 ms and the London edge RTT is ~5 ms. Assume TLS 1.3 (1 RTT handshake) and 40 ms of origin app time. We trace time-to-first-byte (TTFB) for the same 200 KB asset three ways.

Read the punch line: a hit is ~18× faster than direct serving (15 ms vs 280 ms), and even a miss beats direct because the edge amortizes the expensive TLS handshake over a persistent, pre-warmed origin connection instead of paying three fresh cross-ocean round trips. If this asset gets 10,000 London requests during its TTL, exactly one pays the 135 ms miss and 9,999 pay 15 ms — a hit ratio of 99.99%. Origin egress and CPU drop by the same factor, which is often the bigger win than latency. Put it in throughput terms: 10,000 req/s at a 99% hit ratio leaves the origin only ~100 req/s, and at 50 KB per object the edge absorbs roughly 9,900 × 50 KB ≈ 500 MB/s of egress the origin never serves — the offload, not the millisecond, is usually what justifies the CDN.

Close the loop with money, because egress is priced per boundary crossing — the same hit ratio that saves the milliseconds saves the egress bill. That 500 MB/s the edge absorbs is 495 × 10⁶ B/s × 2.592 × 10⁶ s/month ≈ 1.28 PB ≈ 1,280,000 GB per month. Priced three ways (every rate approximate — [VERIFY: current provider pricing — rates change]):

So at this scale the CDN plausibly pays for itself several times over in egress alone, before counting the origin fleet it lets you shrink. The arithmetic inverts at the other extreme: for a low-traffic or largely uncacheable workload, the hit ratio is poor, the origin still serves most bytes, and the CDN's per-request fees plus config surface exceed the little egress it saves — that is precisely the “direct serving wins” regime below.

Freshness: TTL, Cache-Control, and invalidation

The edge decides how long to keep an object from origin response headers. The knobs a systems engineer actually sets:

• Cache-Control: public, max-age=60, s-maxage=86400 — max-age governs the browser; s-maxage governs shared caches (the CDN). Splitting them lets the browser recheck often while the edge holds the object for a day.
• stale-while-revalidate=600 — serve the stale copy instantly and refresh in the background, so a TTL expiry never stalls a user.
• ETag / Last-Modified + conditional If-None-Match — on revalidation the origin can answer 304 Not Modified (headers only, no body), cheap even on a miss-that-wasn't.

Pushing a change before TTL expiry needs invalidation, and there are two philosophies:

• Explicit purge — call the CDN's purge API by URL or by cache tag (e.g. purge everything tagged product:42). Correct but not instant: propagation across hundreds of PoPs takes seconds, and mass purges hammer your origin as every edge simultaneously re-pulls (a purge stampede).
• Versioned URLs / cache busting — ship app.9f3c1.js instead of app.js. The URL is the version, so you never purge; old and new coexist and you set max-age=31536000, immutable. This is the preferred pattern for static assets because it sidesteps invalidation entirely.

Pitfalls

• Caching a personalized response. An edge caches a logged-in user's dashboard (it carried a Set-Cookie but a permissive Cache-Control) and serves it to the next visitor — a data leak. Rule: mark private/personalized responses Cache-Control: private, no-store and only cache what is safe to share.
• Cache key too coarse (or too fine). Forgetting Vary: Accept-Encoding can serve a gzip body to a client that didn't ask for it. Conversely, letting a random query param (?utm_source=…) into the key fragments the cache into thousands of near-duplicate entries, collapsing the hit ratio.
• Cold-cache / thundering herd on origin. A viral spike hits objects no PoP holds yet; every edge pulls simultaneously and the origin melts. Mitigate with origin shield (a designated mid-tier PoP that collapses concurrent misses into one origin fetch) and request coalescing.
• TTL too long on truly dynamic data. A stock price or inventory count cached for an hour shows stale numbers. Match TTL to the data's real change rate, or use s-maxage=0 + revalidation.
• Assuming the CDN is up. The edge sits in your critical path; a CDN provider outage takes your whole site down even though your origin is healthy. Have a DNS failover or multi-CDN plan for tier-1 services.

When to use a CDN — and when direct serving wins

Reach for a CDN when the signals point that way: a geographically spread audience; static or cacheable content (images, video, JS/CSS, downloads) with a high read-to-write ratio; traffic spikes you can't provision origin for; or an origin whose egress bandwidth/CPU is the bottleneck. The CDN converts a latency problem and an origin-load problem into a cache-hit-ratio problem.

Prefer direct origin serving when the content is per-request dynamic and uncacheable (a bank statement, a search over live inventory), the audience is single-region and close to your origin (a metro-local internal tool), traffic is low enough that the origin never sweats, or strict data-residency rules forbid copies sitting in third-party PoPs. Here a CDN adds cost, an extra hop, a purge workflow, and a new dependency for a latency win that rounds to zero.

Trade-offs vs the named alternatives

• vs. direct serving: you gain metro-local latency and origin offload; you pay in per-GB and per-request cost, config surface (cache keys, TTLs, purge), a hard dependency on the provider, and cache-coherence bugs that simply don't exist when one server owns the truth.
• vs. a single regional cache (e.g. Varnish/Nginx in front of origin): a reverse-proxy cache gives you offload and revalidation logic without global distribution — cheaper and fully in your control, but it does nothing for a Tokyo user hitting a Virginia box. Choose the in-house cache when your users are regional; choose the CDN when they're global.

Choose a CDN when you serve cacheable content to a global or bursty audience and origin load/latency is real. Prefer direct serving (optionally behind a regional reverse-proxy cache) when content is dynamic/personalized, the audience is local, or a third-party edge copy is a compliance or dependency risk. The common production answer is hybrid: CDN for static assets and cacheable API GETs, origin-direct for authenticated, write, and personalized paths.

Takeaways

• The CDN win is one substitution — an 80 ms cross-region RTT replaced by a 5 ms metro RTT — multiplied across every request that hits cache; a 99%+ hit ratio also slashes origin egress and CPU.
• Routing to the nearest edge is anycast (network picks the PoP) or DNS steering (DNS returns a nearby IP); the object then resolves as a hit (served locally) or a miss (origin pull, then stored per Cache-Control).
• Freshness is engineering, not magic: tune s-maxage/stale-while-revalidate, and prefer versioned URLs over purge APIs so you never fight invalidation.
• A CDN is worth its cost and dependency only for cacheable content and a spread/bursty audience; local, dynamic, or residency-constrained workloads are better served directly, often behind a plain regional reverse-proxy cache.

Sources: Grokking the System Design Interview (DesignGurus); Cloudflare Learning Center on Anycast, cache hit ratio, and origin shield; Fastly and AWS CloudFront documentation on cache keys, TTL, and invalidation; MDN Web Docs on HTTP Cache-Control, ETag, and stale-while-revalidate; Ilya Grigorik, High Performance Browser Networking (O'Reilly) for RTT/TLS latency figures. Re-authored/Deepened for this guide.