API Gateway Pattern An Example

An API gateway is a reverse proxy that terminates the client's connection at the network edge, applies cross-cutting concerns once (TLS, authentication, rate-limiting), then forwards each request as a fresh HTTP call to one or more internal services and stitches their responses into a single reply — so the client makes one authenticated round trip instead of talking to a dozen services directly.

The word that matters is network. The gateway and the services are separate processes, usually separate machines. A book-detail request does not reach into a UserService object in the same heap; it opens a socket to catalogue-svc:8080, sends bytes, and waits. Everything interesting about the pattern — routing, edge auth, response aggregation, protocol translation, graceful degradation when one backend is down — lives in that gap between processes.

Why the naive in-process version misses the point

A gateway that holds live service references and does return userService.getUserDetails(id) is just a facade object: one method call in one JVM, dispatched by the runtime. It shows none of the pattern's reasons to exist. There is no hostname to resolve, no token to verify, no timeout when a downstream hangs, no way to fan out to three services and merge the results, and no single choke point where you'd actually put rate-limiting. If your "gateway" compiles into the same binary as the services it calls, you have not built a gateway — you've built a class. The real thing is defined by the HTTP boundary it sits on.

One request, traced end to end

A mobile app renders a book-detail screen. It sends one call: GET /api/books/9780134685991 with header Authorization: Bearer eyJhbGci…. The screen needs three facts that live in three different services — the title, whether copies are in stock, and the average rating. Here is what the gateway actually does with real values:

Because 4a–4c run concurrently, the fan-out costs max(22,15,31)=31 ms, not the 68 ms sum. That ~32 ms is the gateway's internal, backend-facing work; true client-perceived latency must add the WAN round trip the client pays to reach the edge (often ~100 ms on cellular). The gateway's latency win is not erasing the WAN leg — it is collapsing three TLS handshakes, three auth checks, and three separate cellular round trips into one.

The counterfactual (no gateway). The mobile app would open three separate HTTPS connections over cellular — each a fresh TLS handshake plus its own ~100 ms round trip. Even issued in parallel, the device pays three connection taxes and must orchestrate the merge. Serialized, that is ≈ 300 ms+. It would also have to know three hostnames, embed the auth token three times, and re-implement retry/timeout logic in the client. The gateway collapses that to one authenticated round trip and moves the merge work server-side onto a fast internal network.

The gateway handler (correct version)

Note what makes this a gateway and not a facade: http.NewRequestWithContext + DefaultClient.Do — real network calls to remote services — plus edge auth, a shared timeout via context, concurrent fan-out, and graceful degradation (catalogue is required; stock and rating are best-effort).

// Go 1.22. TokenVerifier / RateLimiter are interfaces implemented elsewhere.
func fetchJSON(ctx context.Context, url, token string, out any) error {
    req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
    if err != nil {
        return err
    }
    req.Header.Set("Authorization", "Bearer "+token) // gateway's own service credential
    resp, err := http.DefaultClient.Do(req)           // <- a real network hop
    if err != nil {
        return err
    }
    defer resp.Body.Close()
    if resp.StatusCode != http.StatusOK {
        return fmt.Errorf("upstream %s -> %s", url, resp.Status)
    }
    return json.NewDecoder(resp.Body).Decode(out)
}

// GET /api/books/{isbn}
func (g *Gateway) GetBook(w http.ResponseWriter, r *http.Request) {
    // (1) Cross-cutting concerns, enforced once at the edge.
    claims, err := g.auth.Verify(r.Header.Get("Authorization"))
    if err != nil {
        http.Error(w, "unauthorized", http.StatusUnauthorized)
        return
    }
    if !g.limiter.Allow(claims.UserID) {
        http.Error(w, "rate limited", http.StatusTooManyRequests)
        return
    }

    isbn := r.PathValue("isbn")
    ctx, cancel := context.WithTimeout(r.Context(), 300*time.Millisecond)
    defer cancel() // one budget bounds ALL downstream calls

    // (2) Fan out over the network, concurrently.
    var (
        cat struct{ Title string `json:"title"` }
        inv struct{ Available int `json:"available"` }
        rev struct{ Rating float64 `json:"rating"` }
        wg  sync.WaitGroup
    )
    catErr := make(chan error, 1)
    best := func(url string, out any) { defer wg.Done(); _ = fetchJSON(ctx, url, g.svcToken, out) }

    wg.Add(3)
    go func() { defer wg.Done(); catErr <- fetchJSON(ctx, g.catalogue+"/books/"+isbn, g.svcToken, &cat) }()
    go best(g.inventory+"/stock/"+isbn, &inv)
    go best(g.reviews+"/rating/"+isbn, &rev)
    wg.Wait() // Wait establishes happens-before: cat/inv/rev are safe to read now

    // (3) Degrade gracefully: catalogue required, stock/rating optional.
    if err := <-catErr; err != nil {
        http.Error(w, "book unavailable", http.StatusBadGateway)
        return
    }
    _ = json.NewEncoder(w).Encode(BookView{
        ISBN: isbn, Title: cat.Title, Available: inv.Available, Rating: rev.Rating,
    })
}

The same shape in Java 17 with the built-in async HttpClient — the aggregation is the point, so the three calls launch together and we join on all of them:

HttpClient http = HttpClient.newHttpClient();
var catalogue = getJson(http, base + "/catalogue/books/" + isbn, svcToken);
var inventory = getJson(http, base + "/inventory/stock/" + isbn, svcToken);
var reviews   = getJson(http, base + "/reviews/rating/"  + isbn, svcToken);
CompletableFuture.allOf(catalogue, inventory, reviews).join(); // wait for all three
String merged = merge(catalogue.join(), inventory.join(), reviews.join());

static CompletableFuture<String> getJson(HttpClient c, String url, String token) {
    HttpRequest req = HttpRequest.newBuilder(URI.create(url))
        .header("Authorization", "Bearer " + token)
        .timeout(Duration.ofMillis(300))
        .GET().build();
    return c.sendAsync(req, HttpResponse.BodyHandlers.ofString())
            .thenApply(HttpResponse::body);
}

In production you rarely hand-write routing for pass-through endpoints — Spring Cloud Gateway, Kong, Envoy, or AWS API Gateway declare routes in config. But aggregation endpoints like this one are exactly where you drop into code, and increasingly that code moves into a per-client Backend-for-Frontend (see trade-offs below).

Fan-out over a list: cap concurrency, prefer a batch endpoint

The three-call example above fans out to distinct services, so unbounded goroutines are fine — there are only three. The danger is the N+1-over-the-network shape: rendering a 40-item list by looping one GET /stock/{isbn} per row. Do it serially and you pay 40 × 15 ms ≈ 600 ms; do it with 40 simultaneous goroutines and you open 40 sockets to inventory at once, and a spike of such requests can exhaust its connection pool. The fix is a hard concurrency cap — a bounded worker pool. In Go the canonical primitive is errgroup with SetLimit:

// go get golang.org/x/sync/errgroup
func (g *Gateway) stockFor(ctx context.Context, isbns []string) (map[string]int, error) {
    eg, ctx := errgroup.WithContext(ctx)
    eg.SetLimit(8) // never more than 8 concurrent sockets to inventory
    var mu sync.Mutex
    out := make(map[string]int, len(isbns))
    for _, isbn := range isbns {
        isbn := isbn // capture per iteration
        eg.Go(func() error {
            var v struct{ Available int `json:"available"` }
            if err := fetchJSON(ctx, g.inventory+"/stock/"+isbn, g.svcToken, &v); err != nil {
                return err
            }
            mu.Lock(); out[isbn] = v.Available; mu.Unlock()
            return nil
        })
    }
    return out, eg.Wait() // blocks until all done; first error cancels ctx
}

(A buffered channel used as a counting semaphore — sem := make(chan struct{}, 8), acquire before each call, release in a defer — is the dependency-free equivalent.) With the cap at 8, the 40 items run in ⌈40/8⌉ = 5 waves of ~15 ms ≈ 75 ms — 8× faster than serial, without the 40-socket storm.

But the real win is upstream of concurrency: ask for a batch endpoint. If inventory exposes GET /stock?isbns=…, 40 round trips collapse to one (~20 ms) — fewer sockets, one set of headers, and the downstream can answer with a single index scan. Rule of thumb: cap concurrency when you're forced to loop; design a batch endpoint when you control the downstream.

Hedged requests for the tail. When a required call has a fat tail (a slow replica, a GC pause), issue a second attempt to another backend after roughly the p95 latency and take whichever response returns first, cancelling the loser. You trade a small amount of extra load (only the ~5% that cross p95 get a twin) for a materially shorter p99 — the standard tail-tolerance technique from the Google SRE practice of tied/hedged requests.

Pitfalls

• Single point of failure & single choke point. Every request flows through the gateway; if it dies, the whole API dies. It must be horizontally scaled behind a load balancer and health-checked. A slow gateway slows everything.
• Latency-tail amplification on fan-out. A request that touches N services is as slow as the slowest of the N (the tail), and fails if any required one fails. Merging 5 backends where each has a 1% error rate yields a ~5% endpoint error rate unless you make calls optional or add fallbacks. Always set a per-call timeout and a total budget (the context above).
• The "god gateway." Teams keep adding business logic — price calculations, entitlement rules, response reshaping — until the gateway becomes a distributed monolith that every team must coordinate to deploy. Keep it to routing, auth, and thin aggregation; push domain logic into services.
• Fan-out storms / N+1 over the network. Aggregating a list ("20 books, each needs stock") by looping one HTTP call per item hammers downstreams. Prefer batch endpoints (/stock?isbns=…) and cap concurrency.
• Timeout & retry misconfiguration. If the gateway retries on timeout while the downstream is merely slow (not failed), you multiply load on an already-struggling service. Use retry budgets, jittered backoff, and circuit breakers.
• Auth done twice or not at all. The gateway verifies the client token, but services must still trust only the gateway (mTLS or a signed service token) — otherwise anyone who reaches the internal network bypasses auth entirely.

When to use it — and when not

Reach for a gateway when you have several services behind one public API, multiple clients that shouldn't each re-implement auth/TLS/rate-limiting, and screens that need data from more than one service in a single call. The concrete signals: clients are making 3+ calls to paint one view; auth/observability logic is copy-pasted into every service; you're exposing internal service topology (hostnames, ports) to the public internet.

Trade-offs and named alternatives:

How a senior decides: Choose a single gateway when you have one dominant client and a modest service count. Prefer a BFF per client when a mobile app and a web app want very different response shapes and a shared gateway keeps sprouting if client==mobile branches. Use a service mesh alongside (not instead of) the gateway — the mesh secures and observes internal hops while the gateway owns the public edge. Skip the gateway entirely for a two-service system or an internal-only tool where the added hop and operational burden buy you nothing.

Takeaways

• The pattern is defined by the process/network boundary: real HTTP calls to remote services, not in-heap object delegation.
• Its value is concentrating cross-cutting concerns (auth, TLS, rate-limit) once and turning many client calls into one aggregated response over a fast internal network.
• Fan out concurrently under one timeout budget, and mark each dependency required or best-effort so one slow backend doesn't sink the request.
• Keep it thin. A gateway that grows business logic becomes a distributed monolith; when clients diverge, split into BFFs, and let a service mesh handle internal traffic.

Re-authored and deepened for this guide. Sources: Chris Richardson, Microservices Patterns (Manning, 2018) and microservices.io (API Gateway / Backend-for-Frontend patterns); Sam Newman, Building Microservices, 2nd ed. (O'Reilly, 2021); the NGINX "Building Microservices: Using an API Gateway" article; and the Go net/http and Java java.net.http standard-library docs. Go example compiled and vetted under Go 1.25; the Java snippet targets Java 11+ HttpClient.