Security Considerations

Service-registry security works by making every write (register/deregister) and every read (query) pass an identity check the registry itself enforces: the caller presents a credential — an ACL token or a client TLS certificate — and the registry compares the service name being touched against a policy bound to that credential before it will mutate the routing table or return the map of who lives where. If that gate is missing or misconfigured, discovery degrades from "the place services find each other" into "the place an attacker reroutes your traffic."

Why the registry is the sensitive part

A registry is a live, authoritative index of every instance's address and health. Two properties make it a prize target. First, it is write-authoritative: whatever it says is where payments lives becomes where clients send card numbers. Second, it is a topology map: a single unauthenticated read enumerates every internal service, IP, port, and health state — reconnaissance an attacker would otherwise spend weeks assembling. Encrypting the channel (TLS) hides the traffic from eavesdroppers but does nothing to stop a caller who is allowed to talk to the registry from lying to it. Encryption and authorization are orthogonal; you need both.

Trace: a service-spoofing attack, with and without an ACL gate

A pod is compromised on the cluster and can reach the Consul HTTP API on :8500. It wants traffic destined for payments. The real payments instance is at 10.0.2.10:8443; the attacker's listener is at 10.0.9.99:8443. Follow the same request through two configurations.

What the ACL policy actually gates

The gate is not "is this caller authenticated" — it is "is this caller allowed to write this exact service name." A Consul ACL policy is explicit rules over named resources. The policy issued to the real payments deployment says: you may register/deregister payments, and you may read every service (so it can discover its own dependencies), and nothing else.

# policy: payments-svc.hcl
service "payments" {
  policy = "write"      # register / deregister ONLY this name
}
service_prefix "" {
  policy = "read"       # may discover any service (least-priv variant below)
}
node_prefix "" {
  policy = "read"
}

You bind that policy to a token and hand the token only to the payments workload (a Kubernetes Secret or a Vault-issued lease), never checked into an image:

consul acl policy create -name payments-svc -rules @payments-svc.hcl
consul acl token create -description "payments" -policy-name payments-svc
# -> SecretID: 8f4e...  (mounted at /var/run/secrets/consul-token)

The attacker cannot forge this: the SecretID is a 128-bit random value the registry stores server-side and matches on every request. Absent that exact token, the service "payments" { write } rule never fires, so step 2 in the trace returns 403. This is the whole mechanism — a name-scoped write grant that only the legitimate owner of that name holds.

Least privilege, sharpened

The service_prefix "" { read } rule above still lets any authenticated service read the whole map. Tighten it: a frontend that only calls accounts and orders gets a policy that names exactly those, so a compromised frontend token still cannot enumerate payments or the internal fraud-scoring service. This turns one breach into a bounded blast radius instead of a full topology dump.

# policy: frontend-svc.hcl — read only its two dependencies
service "accounts" { policy = "read" }
service "orders"   { policy = "read" }
service "frontend" { policy = "write" }   # register itself

Pitfalls

• default_policy = allow with ACLs "enabled." This is the classic false sense of security. ACLs are on, tokens exist — but the default is allow, so any request without a matching deny rule succeeds. This is exactly the left column of the trace. The only safe posture is default_policy = deny; then absence of a grant means rejection.
• Over-privileged anonymous token. Consul evaluates tokenless requests under the built-in anonymous token. If that token carries any write, ACLs are theater — the attacker just omits the header. Bind the anonymous token to a read-nothing/write-nothing policy explicitly.
• Bearer tokens leak and replay forever. A SecretID in a log line, an env dump, or a stack trace is a permanent skeleton key until rotated. Short-lived, Vault-issued tokens (or mTLS certs) shrink that window.
• TLS ≠ authorization. Teams enable HTTPS on :8501 and call it "secured." TLS stops sniffing and MITM on the wire; it does not stop an authorized-to-connect caller from registering a fake instance. Spoofing lives in the authz layer, not the transport layer.
• Health-check spoofing. The attacker registers with a check it controls (a TTL it keeps refreshing, or an HTTP check pointing at its own always-200 endpoint), so the fake instance stays healthy and in rotation indefinitely.
• Eureka has no per-service identity. Basic auth or mTLS on Eureka gates connection, not which name you may claim — one shared credential lets any holder register anything as anything. It cannot express "only payments may register payments." If you need registration authz, that is a reason to reach for Consul/mesh, not Eureka.
• Confusing Kubernetes RBAC with runtime traffic control. K8s RBAC gates the API server — who may create/read Service and EndpointSlice objects. It does not stop a compromised pod from opening a TCP connection straight to payments's ClusterIP, and DNS resolution is unauthenticated inside the cluster. Pod-to-pod trust requires NetworkPolicy plus a mesh (mTLS), not RBAC.

When to use which control (and what it costs)

Registry security is layered; the real decision is where your identity lives.

Choose Consul ACLs when your threat is fake registrations and topology leaks and you can manage tokens. Prefer a service mesh (mTLS/SPIFFE) when a leaked bearer token is unacceptable and you need cryptographic, short-lived, per-call workload identity — accepting the sidecar latency and operational tax. Never rely on network isolation alone: use it under one of the other two, because internal-flat-trust dies at the first compromised pod. And on capacity: put rate limits on the registry's write/query API (it is a shared chokepoint — a registration flood can starve real lookups) and turn on audit logging, because in the deny-column trace it is the audit line that tells you an attack happened at all.

Microservices threat model: three attack paths

Registry spoofing is one path; a complete microservices threat model also covers service-to-service traffic and secret handling. The table below names the attacker, the vulnerable surface, and the control.

Sidecar vs library: where to put the policy

You can implement mTLS, retries, and telemetry in an application library or in a sidecar proxy. The choice is an ownership trade-off.

Zero-trust service mesh checklist

A mesh is not zero trust just because it uses mTLS. Use this checklist during design review.

• Identity is workload-bound — Every pod/service has a short-lived certificate or SPIFFE ID issued by an attested CA, not a long-lived shared secret.
• Default deny — No service can talk to another unless an authorization policy explicitly allows it, even inside the cluster.
• mTLS everywhere — All service-to-service traffic is encrypted and mutually authenticated; plaintext fallback is rejected, not merely discouraged.
• Token scope is minimal — Service tokens and user JWTs carry only the claims needed for the call; broad "service admin" tokens are forbidden.
• Secrets rotate automatically — Certificates and tokens have short TTLs; rotation does not require application redeploy.
• Audit every denied registration and authz failure — The audit stream is monitored, not just collected.
• Runtime policy can be updated centrally — Authorization rules, rate limits, and traffic shifts deploy without changing application code.
• Fallback to plaintext is impossible — NetworkPolicy plus mesh policy blocks direct pod-to-pod bypass.

Secret-leak trace and rotation story

The following trace follows a real-shaped incident: a database password is accidentally logged, detected, and rotated without downtime.

Lessons from the trace: (1) secrets should never be interpolated into log statements — use structured logging and mask known secret patterns; (2) rotation must be routine and fast, not a multi-day migration; (3) a grace period prevents a hard cutover from breaking in-flight connections; (4) the ability to reload without redeploy makes rotation cheap enough to do often.

Rate-limiting and auditing the registry write path

ACLs decide who may register; two operational controls bound how hard the write path can be hit and make abuse visible after the fact. First, a Consul agent's limits block caps how many concurrent HTTP connections a single client may hold open against it (default 200):

# agent configuration — cap concurrent HTTP connections per client
limits {
  http_max_conns_per_client = 100
}

Request-rate limiting is a separate, control-plane concern: Consul throttles read and write requests against the servers through its control-plane-request-limit configuration entry (with permissive and enforcing modes) — there is no per-client-IP rate-limit knob in the agent config, so if you need true per-caller throttling of the registry's HTTP API, put a rate-limiting proxy or load balancer in front of it (see the rate-limiter page below). For visibility, Consul's audit logging — an Enterprise feature — records every HTTP API request, including denied registration writes, as a JSON event; alert on denials against service-registration endpoints. Check the exact configuration shapes against the current Consul documentation (agent limits reference, the control-plane-request-limit config entry, and Enterprise audit logging) — these surfaces have shifted across versions.

Takeaways

• The registry is both write-authoritative (it decides where traffic goes) and a topology map (one read reveals everything) — treat it as tier-0 security infrastructure.
• The gate that stops spoofing is a name-scoped write grant: only the holder of the payments token may register the name payments. Set default_policy = deny or the gate does nothing.
• TLS, ACLs, RBAC, and NetworkPolicy are orthogonal layers — encryption is not authorization, and K8s RBAC does not govern pod-to-pod traffic.
• Bearer tokens authorize whoever holds the string; mTLS/SPIFFE authorizes the workload itself — move up to a mesh when replayable secrets are unacceptable.
• A microservices threat model covers service-to-service identity, token propagation, registry integrity, secret lifecycle, and sidecar bypass — not just perimeter defense.
• Zero trust is a policy default (deny all, explicit allow) plus short-lived workload identity, not a single product you install.

Related pages

• TLS & mTLS — The Handshake, Step by Step — the transport layer that protects registry traffic.
• Securing a System — Defense in Depth — zero-trust and layered controls.
• Designing an API Rate Limiter — rate limiting the registry write/query API.
• Metrics, Logs & Traces — audit logging and anomaly detection.

Sources: HashiCorp Consul documentation — ACL system, security model, and Connect service mesh; the SPIFFE/SPIRE specification and Istio security architecture for workload identity via mTLS; Chris Richardson, Microservices Patterns (Ch. 11, security); Sam Newman, Building Microservices, 2nd ed. (zero-trust and defence-in-depth); Kubernetes docs on RBAC and NetworkPolicy; OWASP guidance on service-to-service authentication. Re-authored/Deepened for this guide.