What is Service Discovery Pattern

What is the Service Discovery Pattern

In a monolith, one function calls another by name and the linker resolves the address at compile time. In a microservices system that guarantee disappears. Your orders service needs to call payments, but payments is not a fixed machine — it is a herd of ephemeral instances that autoscaling spins up and kills, that crash and restart, that get rescheduled onto different hosts by Kubernetes, each landing on a fresh IP and port. Hard-coding 10.0.3.14:8080 into orders is a trap: the instance behind that address will be gone in an hour.

Service discovery is the mechanism that answers one question at runtime: "I want to talk to the payments service — which live instances exist right now, and where are they?" It decouples the logical name of a service from the physical network locations of its instances, so callers work with stable names while the underlying addresses churn freely underneath them.

How it works, precisely

Every service discovery setup has three moving parts. The service registry is a highly-available database of (service-name → list of instance endpoints), plus health metadata. This is the source of truth — Consul, etcd, ZooKeeper, Eureka, or the built-in registry inside Kubernetes (etcd behind the API server).

Registration puts instances into the registry. Two styles exist. In self-registration, the instance calls the registry on startup ("I am payments, I am at 10.0.3.14:8080") and sends periodic heartbeats; miss a few and the registry evicts it. In third-party registration, a separate registrar (the platform — e.g. Kubernetes watching pod readiness probes) registers and deregisters instances on the app's behalf, so application code stays clean.

Discovery is how a caller resolves the name. Again two styles:

• Client-side discovery: the caller queries the registry directly, gets the full instance list, and picks one itself using a load-balancing rule (round-robin, least-connections, zone-aware). Netflix Eureka + Ribbon is the classic example. The client owns the balancing logic.
• Server-side discovery: the caller sends the request to a stable virtual endpoint — a load balancer, a Kubernetes Service ClusterIP, or a service-mesh sidecar. That intermediary queries the registry and forwards to a healthy instance. The client stays dumb; it just hits one address.

Two properties make the registry usable at scale: health checking (only healthy instances are returned, via heartbeats, TTLs, or active probes) and caching (clients cache results for seconds to avoid hammering the registry, trading a little staleness for huge QPS reduction).

A worked scenario with numbers

Picture a checkout flow. The orders service handles 5,000 QPS and calls payments on every request. payments runs 20 instances behind autoscaling; during a flash sale it scales to 60 and back down within minutes, and roughly 3 instances per minute are being rotated (crashes, deploys, rescheduling).

If orders hit the registry on every call, that is 5,000 registry QPS from just one caller — the registry becomes a bottleneck and a single point of failure. Instead, each orders instance caches the instance list for ~30 seconds and refreshes in the background. Registry load drops to a trickle. The cost is bounded staleness: for up to 30 seconds after an instance dies, some clients still hold its address. Health checks with a 10-second heartbeat and a 30-second eviction TTL plus client-side retry-on-another-instance keep the blast radius tiny — a failed call to a dead .14 just retries .27, adding maybe 5–20 ms of retry latency to a fraction of requests rather than failing the checkout.

In Kubernetes this is mostly invisible: orders calls http://payments, DNS resolves the Service ClusterIP, and kube-proxy (or a mesh sidecar) load-balances across the current healthy pods, which the control plane keeps in sync with readiness probes. The pattern is the same — the platform just runs the registry and the server-side balancer for you.

Trade-offs: when to use, when not, versus the alternatives

Client-side vs server-side discovery. Client-side (Eureka/Ribbon) gives you smart, application-aware balancing (zone affinity, weighted routing) and removes an extra network hop and a component to run — but it couples every client to registry-specific logic and needs a discovery library per language, which hurts in a polyglot fleet. Server-side (K8s Service, load balancer, mesh) keeps clients trivial and language-agnostic and centralizes routing policy, at the cost of an extra hop and a piece of infrastructure that must itself be highly available.

Versus DNS alone. Plain DNS is a form of server-side discovery, but classic DNS is weak here: TTL caching by resolvers means stale records linger, and vanilla A-records carry no health or load info. It works when instance sets change slowly; it struggles with fast-churning ephemeral instances unless you add short TTLs and health-aware DNS (as SkyDNS/CoreDNS do).

Versus hard-coded config / a static load balancer VIP. Perfectly fine when you have a small, stable set of long-lived hosts. Reach for full service discovery only when instances are dynamic and numerous — autoscaling, frequent deploys, container orchestration. When NOT to use it: a monolith or a handful of fixed VMs behind one load balancer — a registry there is complexity with no payoff.

Versus a service mesh. A mesh (Istio, Linkerd) is service discovery plus mTLS, retries, circuit breaking, and observability, implemented with sidecars. Adopt it when you need those cross-cutting concerns fleet-wide; skip it if plain discovery already meets your needs, because the sidecar tax (latency, memory, operational surface) is real.

Pitfalls an interviewer probes

• The registry is a single point of failure. If it dies, can new lookups happen? Good answer: run it as a replicated cluster (etcd/Consul use Raft), and have clients cache last-known-good instances so existing traffic survives a registry outage — availability degrades gracefully rather than failing hard.
• Stale entries and zombie instances. How fast do dead instances leave the registry, and what happens to a client that calls one? Expect a discussion of heartbeat interval vs eviction TTL, and client-side retries with a different instance plus circuit breakers so a dead endpoint does not cascade.
• CAP trade-off in the registry. ZooKeeper/etcd are CP (they may refuse reads during a partition to stay consistent); Eureka is deliberately AP (it serves possibly-stale data during a partition to stay available). Which do you want for discovery? Usually AP — a slightly stale instance list beats no list at all.
• Thundering herd / self-registration storms. A mass restart floods the registry with registrations and heartbeats. Mitigate with jittered heartbeats, client caching, and rate limits.
• Who owns registration? Confusing self-registration with third-party registration, or forgetting deregistration on shutdown (leaving zombies), is a common slip.

Client-side vs server-side discovery decision table

Health-check + TTL expiry trace

These ticks match the Eureka config below: a 10 s heartbeat, a 30 s eviction TTL (three missed beats), plus a ~30 s client-cache refresh. The bounded staleness window is the sum of the eviction TTL and the client cache TTL — here up to ~60 s — so tune them, and lean on retries, to cover the gap without hammering the registry.

Config snippets: Consul, Eureka, Envoy

These are representative fragments, not production manifests, but they show the shape each system expects (note Consul's agent API uses PUT for registration — a common trip-up).

Consul service registration (HTTP API)

PUT /v1/agent/service/register
{
  "ID": "payments-14",
  "Name": "payments",
  "Tags": ["v1", "us-east"],
  "Port": 8080,
  "Check": {
    "HTTP": "http://10.0.3.14:8080/health",
    "Interval": "10s",
    "Timeout": "5s"
  }
}

Spring Eureka client (application.yml)

eureka:
  client:
    serviceUrl:
      defaultZone: http://eureka:8761/eureka/
  instance:
    leaseRenewalIntervalInSeconds: 10
    leaseExpirationDurationInSeconds: 30

Envoy cluster with EDS

clusters:
  - name: payments
    connect_timeout: 0.25s
    type: EDS
    eds_cluster_config:
      eds_config:
        path: /etc/envoy/endpoints.yaml

The common theme: register, heartbeat, and let callers resolve names to live endpoints.

Trap: split-brain and stale registry

A network partition can split the registry cluster itself. In a CP registry (etcd, ZooKeeper, Consul with consistency mode), a minority partition stops accepting writes so the instance list stays consistent but may become unavailable for updates. In an AP registry (Eureka), each partition keeps accepting registrations, so after healing the two sides may disagree on which instances are alive.

The practical defense is the same for both: clients cache last-known-good endpoints, use short TTLs, and retry on a different instance. A stale registry entry should cause a retry, not an outage. Never trust the registry to be perfectly consistent; design for bounded staleness.

Key takeaways

• Service discovery decouples a stable logical service name from the churning physical addresses of its instances, via a health-aware registry that instances register into and callers look up at runtime.
• Choose client-side discovery for smart, app-aware balancing without an extra hop; choose server-side (K8s Service, LB, mesh) for language-agnostic clients and centralized routing — most container platforms give you the latter by default.
• Use it when instances are dynamic and numerous (autoscaling, frequent deploys); skip it for a monolith or a small set of fixed hosts where a static VIP suffices.
• The hard parts are operational: keep the registry highly available and cached, tune heartbeat vs eviction TTL, and pair discovery with retries and circuit breakers so stale or dead entries never fail a request outright.
• Bounded staleness is a budget you tune: eviction TTL + client cache TTL bounds how long a dead instance can receive traffic (~60 s in the worked config) — cover the gap with retries, never with a shorter heartbeat alone.