Circuit Breaker Pattern An Example

Let’s implement a basic Circuit Breaker in Java to illustrate the core algorithm. We won’t use any external libraries – just a simple class to demonstrate the state logic:

public class CircuitBreaker {
    private enum State { CLOSED, OPEN, HALF_OPEN }
    private State state = State.CLOSED;
    private int failureCount = 0;
    private final int failureThreshold;
    private final long openTimeout;       // how long to stay open before trying half-open
    private long lastFailureTime = 0;

    public CircuitBreaker(int failureThreshold, long openTimeout) {
        this.failureThreshold = failureThreshold;
        this.openTimeout = openTimeout;
    }

    public synchronized <T> T call(Callable<T> remoteCall) throws Exception {
        // 1. If breaker is OPEN, check if timeout has passed to allow a half-open trial
        if (state == State.OPEN) {
            long timeSinceFailure = System.currentTimeMillis() - lastFailureTime;
            if (timeSinceFailure < openTimeout) {
                // Still within open timeout, reject the call fast
                throw new IllegalStateException("Circuit is OPEN. Failing fast without calling remote service.");
            } else {
                // Timeout passed, move to half-open and allow a trial call
                state = State.HALF_OPEN;
            }
        }

        // 2. Try to invoke the remote call (allowed if state is CLOSED or HALF_OPEN)
        try {
            T result = remoteCall.call();
            // Call succeeded. Reset failure count and close the circuit (if it was half-open, now fully close it).
            failureCount = 0;
            state = State.CLOSED;
            return result;
        } catch (Exception e) {
            // Call failed. Increase the failure count.
            failureCount++;
            lastFailureTime = System.currentTimeMillis();
            if (failureCount >= failureThreshold) {
                // Too many failures, trip the circuit to OPEN state.
                state = State.OPEN;
            }
            throw e;  // Propagate the exception (or could return a fallback here)
        }
    }
}

One simplification here is a trap if you copy it: synchronized wraps the remote call itself, so while one call is in flight every other caller — even ones that should fail fast against an OPEN breaker — blocks at the lock before it can so much as check the state. The breaker becomes a serializer: maximum throughput through it is 1/latency (12.5 req/s at 80 ms calls, and near zero while one call hangs toward a timeout) — it re-creates exactly the thread pile-up the pattern exists to prevent. Production implementations hold a lock (or use atomics — e.g. an AtomicReference for state plus LongAdder counters) only for the state check and the outcome update, and execute the remote call outside any lock; Resilience4j's state machine works exactly this way.

In the code above, CircuitBreaker tracks the number of failures and the current state. The call method wraps the execution of a remote operation (passed as a Callable<T>). Here’s how it works:

1. Before calling the remote service, it checks the state. If the circuit is OPEN, the code checks if the open timeout has elapsed. If not (meaning it's too soon to retry), it fails fast by throwing an exception (IllegalStateException in this case) instead of calling the remote service. If the timeout has elapsed, the breaker moves to HALF_OPEN state, allowing one trial call to proceed.

2. Executing the call: If the state is Closed or Half-Open, it attempts the remoteCall. This is where your actual remote service logic would run.

3. After the call returns: If the call succeeds, we reset the failure counter and set the state to CLOSED. (In Half-Open, a success means the service is back, so we fully close the circuit again. In Closed, a success just keeps it closed and can reset any past failure count.) If the call throws an exception or times out, we increment the failure count and record the time of failure. If the failures have reached the failureThreshold, the circuit switches to OPEN (tripped). The exception is rethrown to the caller – in a real system, the caller might catch this and fall back to some default behavior.

Using this CircuitBreaker is straightforward. For example:

CircuitBreaker cb = new CircuitBreaker(3, 10000); // trip after 3 failures, 10s timeout
try {
    String data = cb.call(() -> unreliableService.getData());
    // use data...
} catch (Exception e) {
    // Handle fallback logic because service call failed or circuit is open
    useCachedData();
}

In this example, if unreliableService.getData() fails 3 times in a row, the circuit breaker will open. Further attempts within 10 seconds will immediately throw an exception without calling getData() at all – the circuit breaker is preventing more failures. After 10 seconds, the next call will be allowed to try the service again (half-open). If that next call succeeds, the circuit closes and things go back to normal. If it fails, the circuit re-opens and will block calls for another 10 seconds.

This is a simplistic implementation to demonstrate the concept. In production, a circuit breaker might be more sophisticated – locking only state transitions instead of the whole call (this version is thread-safe, but by holding the lock across the remote call it serializes every caller, as flagged above), using a rolling window of failure metrics, or integrating with monitoring systems – but the core idea is the same.

Interactive walkthrough

Predict the next step in the circuit-breaker scenario.

What the simple example omits

The code above illustrates the state machine, but a production breaker adds three things:

• Rolling window. A counter that never decays will trip on a single failure a week later. Real implementations use a sliding time window or a bucketed failure count.
• Half-open probe policy. In the example, one success closes the breaker. In production you usually require several probe successes before closing, and you limit the number of concurrent probes to one.
• Distributed state. Each instance has its own breaker. If you have 10 app servers and only one sees failures, the breaker may never trip cluster-wide. Service meshes and libraries like Resilience4j can share state through a data store or gossip.

When NOT to wrap a call in a breaker

• The result is required synchronously. "Did the payment succeed?" cannot be replaced by a fallback; a breaker that fails fast still leaves the checkout incomplete.
• The fallback would violate correctness. Returning a cached inventory count during a flash sale can cause overselling.
• The call is in-process and cheap. Inside one process, normal exception handling is usually enough.
• The dependency is already isolated. If a slow call has its own dedicated thread pool and cannot starve anything else, a breaker is less urgent.

Tuning the knobs

Three numbers control behavior:

• Failure threshold — set it high enough to ignore normal blips. If the downstream error rate is normally 0.5%, a threshold of 5 in 10 seconds will trip constantly.
• Open timeout — should match the dependency's recovery time. Too short and you hammer a still-down service; too long and you delay recovery.
• Half-open probes — allow one probe at a time. Multiple concurrent probes can overload a service that is just beginning to recover.

Pairing with retries and bulkheads

A breaker is not a retry policy. Retries recover from transient failures; breakers stop sending load during sustained failures. Use them together: retry a call a small number of times with backoff, and only count the final failure toward the breaker. Combine with a bulkhead so that while the breaker is open, the caller's threads are not tied up waiting.

Takeaways

• The simple three-state machine is correct, but production needs rolling windows, probe policies, and often shared state.
• Use breakers for cross-process/cross-service calls where a fallback exists.
• Avoid breakers when the caller must have the result or when a fallback would violate correctness.
• Tune thresholds against normal error rates, not against zero; monitor trip frequency and recovery time.

The threshold decision the example glosses over: count vs rate, and where they cross

The new CircuitBreaker(3, 10000) above trips on an absolute count — 3 failures. That is the right choice at low traffic and a latent outage at high traffic, and the switchover point is computable, not a matter of taste. Two threshold styles compete:

• Count-based — trip after C failures (the example does this with no window at all: its counter only resets on success, so it is even more volume-fragile than a windowed count). Simple, and it reacts on the very first C failures with no warm-up.
• Rate-based — trip when the failure percentage crosses a threshold over a rolling window, but only once a minimum number of calls has accumulated (e.g. Resilience4j's failureRateThreshold + minimumNumberOfCalls). Robust to volume, but blind until the window fills.

Why a fixed count breaks at high volume. A count threshold C is only meaningful relative to how many calls flow through the window. The normal-noise failures in one window are baseline_error_rate × λ × window, where λ is requests/second. (Careful: the example's 10000 ms is its open-state cooldown, not a failure-counting window — the example has no window, so pick the observation window explicitly.) Take a 10 s observation window, a benign baseline error rate of 1%, and a slightly more forgiving C = 5:

• At λ = 10 req/s: 0.01 × 10 × 10 = 1 expected failure per window. A trip count of 5 sits at a 5% instantaneous error rate — comfortably above noise, and a genuinely 50%-down dependency produces ~50 failures/window and trips almost instantly. Count works, and reacts fast.
• At λ = 500 req/s: 0.01 × 500 × 10 = 50 expected failures per window on a perfectly healthy service. A trip count of 5 is now far below the normal noise floor — the breaker flaps open constantly against a dependency that is fine. Count is broken; you must switch to rate.

The crossover. A fixed count C stops being safe the moment normal-noise failures per window reach it: baseline × λ × window ≥ C. Solving for the traffic where these numbers cross:

λ* = C / (baseline × window) = 5 / (0.01 × 10) = 50 req/s

Below ~50 req/s a small absolute count is the better tool — it is simpler and reacts on the first few failures, whereas a rate window with minimumNumberOfCalls = 100 would take 100 / λ seconds just to start evaluating (20 s at 5 req/s — an eternity during an outage). Above ~50 req/s the count threshold drowns in noise and you must move to a percentage threshold with a minimum-volume floor. The number moves with your inputs — halve the baseline error rate and λ* doubles to 100 req/s, and with the example's own C = 3 it drops to 3 / (0.01 × 10) = 30 req/s — but the rule is fixed: compute C / (baseline × window); if your real traffic is above it, a count threshold will false-trip, so switch to rate.