Advantages and disadvantages of using API gateway

Advantages of an API gateway

An API gateway sits in front of your backend services as the single entry point for client traffic. Centralizing cross-cutting concerns there — rather than repeating them in every service — is the whole value proposition. The main benefits:

• Improved performance. Response caching turns a backend round trip (tens of milliseconds — 30.8 ms in the worked example below) into a sub-millisecond cache hit for cacheable reads, and connection pooling amortizes TCP + TLS setup across many requests instead of paying it per call.
• Simplified system design. One entry point for many services means clients don't need to know internal topology, and operators have one place to observe and control traffic instead of N places.
• Centralized, enhanced security. Authentication, authorization, and TLS termination happen once at the edge instead of being re-implemented — and re-audited — in every microservice.
• Improved scalability and fault tolerance. After deciding which service owns a route, the gateway balances requests across that service's replica pool (upstream load balancing), with health checks taking unhealthy instances out of rotation automatically — so adding backend replicas, or losing one, needs no client change. Note the scoping: choosing among different services is routing, the gateway's own job; spreading load across interchangeable replicas of one service is what it does only within the resolved upstream pool.
• Better monitoring and visibility. Because all traffic passes through one chokepoint, the gateway is a natural place to collect uniform metrics, traces, and access logs across every service.
• Simplified client integration. Clients talk to one consistent interface instead of juggling per-service quirks, auth schemes, and hostnames.
• Protocol and data-format translation. The gateway can bridge HTTP to gRPC, or JSON to XML, so backend teams can choose internal protocols freely without breaking external clients.
• API versioning and backward compatibility. The gateway can route /v1 and /v2 to different backend deployments, letting teams evolve services without breaking existing consumers.
• Consistent error handling. One place normalizes error shapes and status codes instead of every service inventing its own error contract.

Disadvantages of an API gateway

None of this is free. The same chokepoint that gives you leverage also concentrates risk:

• Additional complexity. It's a new component your team must design, deploy, secure, and reason about — one more moving part in every incident.
• Single point of failure. If it goes down without redundancy, everything behind it becomes unreachable, even if every backend service is healthy.
• Added latency. Every request takes an extra hop, and if the gateway does real work — transformation, auth, rate limiting — that hop is not free. See the worked example below for actual numbers.
• Vendor lock-in. Adopting a managed gateway (AWS API Gateway, Apigee, Kong Konnect) couples your routing, auth, and rate-limit configuration to that vendor's model and pricing.
• Cost at scale. Managed gateways typically bill per request or per hour; at high volume this becomes a real line item, not a rounding error.
• Maintenance overhead. Self-hosted gateways need patching, capacity planning, and on-call ownership like any other production service.
• Configuration complexity. Rich feature sets — routing rules, auth policies, transformation, rate limits — multiply the ways a misconfiguration can silently break traffic.

The trade-off in one line: an API gateway exchanges a small, well-understood amount of latency and operational surface for a large reduction in duplicated cross-cutting logic across services. That trade is usually worth it once you have more than a handful of backend services or an external API surface — and usually not worth it for a single monolith with no external clients.

Make the tension concrete with a fleet of 40 services. Pulling authentication into the gateway means one JWT-verification library to patch and re-audit instead of 40 — a real, countable win. The same centralization is the liability: one bad config push can page all 40 teams at once, where before a bug was contained to the one service that shipped it. That is why gateway config deserves the same discipline as application code — progressive/canary rollout of route changes and per-route synthetic probes — so a single edit cannot floor the whole fleet.

Worked example: where does the latency actually go?

"The gateway adds latency" is true but vague. Here is a concrete breakdown for one request that misses the gateway's response cache and has to travel all the way to the backend:

Reading the numbers

Summing the five gateway-side steps — TLS session resumption (0.2 ms), JWT verification (0.5 ms), the rate-limit check (0.6 ms), the cache-miss lookup (0.1 ms), and marshaling the response back to the client (0.4 ms) — gives 1.8 ms of gateway-owned time. Against the 34.6 ms end-to-end total, that is 1.8 / 34.6 ≈ 5.2%, closer to "1 in 20 ms" than "1 in 16." The remaining ~94.8% is network transit and backend processing, which the gateway neither caused nor can optimize away.

The practical implication: on a cache miss, optimizing the gateway itself has a low ceiling. If p99 latency is a problem, profile the backend and the network path first — shaving the gateway's 1.8 ms is diminishing returns next to a 30.8 ms backend call.

Pitfalls to watch for in production

Beyond the textbook disadvantages above, a few failure modes only show up once a gateway is carrying real traffic:

• Retry storms. If the gateway retries failed backend calls without backoff or jitter, a struggling service gets hit even harder right when it can least afford it — turning a partial outage into a full one. Use bounded retries with exponential backoff and jitter, and prefer circuit breakers over blind retries.
• N+1 fan-out. A gateway-level backend-for-frontend endpoint that quietly makes several downstream calls per request multiplies tail latency by the slowest of those calls. Track and budget fan-out depth per route, not just per-hop latency.
• Header and body-size mismatches. Gateways and backends often enforce different maximum header and body sizes. A backend that accepts a 10 MB upload behind a gateway capped at 1 MB fails silently at the edge with a generic 413, which is confusing to debug without correlated logs.
• Caching authenticated responses. The gateway's response cache — listed above as a performance win — turns into a data-leak incident the moment it caches a personalized or authenticated payload without the caller's identity in the cache key: user A's request populates the entry, user B's request hits it and is served A's data. Any route that varies by user must key on (or Vary by) the identity/authorization token, or be marked private/no-store. A cache hit that crosses a trust boundary is a security incident, not a latency saving.
• Buffering large bodies. Some gateways buffer the entire request or response body in memory before forwarding it — useful for signature verification, logging, or transformation, but dangerous for large uploads or downloads. Under concurrent multi-megabyte payloads, this blows up memory and causes timeouts. Stream bodies through the gateway wherever the platform supports it, and set an explicit max-body-size limit so oversized requests fail fast with a clear error instead of exhausting gateway memory under load.

Drill: defend the gateway against a plain ALB

Design-review follow-up: "We have three services. Why an API gateway at all — why not a plain ALB with path-based routing?"

Answer sketch. Start by conceding what the ALB already gives you: TLS termination, path/host-based routing to target groups, health checks, and replica load balancing — for less money and near-zero operational surface. A gateway earns its keep only for the concerns the ALB does not carry: token validation, per-client quotas/rate limits, request/response transformation, and API versioning. The real test is duplication: if each of the three services would otherwise re-implement JWT verification and rate limiting itself, the gateway replaces three copies of that logic (three libraries to patch, three configs to audit) with one. If the three services share no such cross-cutting logic — or it already lives in a shared middleware library — the honest answer is that the ALB wins today, and the gateway is a decision to revisit when the service count or the duplicated edge logic grows.

Sources

Original lesson: "Advantages and disadvantages of using API gateway" — Knowledge Guide, System Design → API Gateway. The latency figures above are an illustrative worked example rather than a measurement from one specific vendor; the order of magnitude for TLS session resumption, JWT verification, and a Redis-backed rate-limit check is consistent with published gateway benchmarks from AWS API Gateway, Kong, and NGINX. The production pitfalls (retry storms, request fan-out, request/response body buffering) draw on Sam Newman, Building Microservices, 2nd ed. (O'Reilly, 2021), and standard reverse-proxy operational guidance from the NGINX and Envoy documentation.