The Architecture of the Saga Pattern

A saga is a sequence of local transactions. Each local transaction runs inside a single service, commits atomically against that service's own database, and then triggers the next step. There is no global lock and no distributed commit protocol holding the whole thing together; the atomicity you get is per step, not across the saga. What replaces the missing global rollback is an explicit, application-level undo: every forward transaction Tₕ that changes state has a paired compensating transaction Cₕ that semantically reverses it.

So a saga is really two things layered together: a forward path (T1, T2, …, Tₙ) that drives the business transaction toward completion, and a backward path (Cₙ₋₁, …, C2, C1) that unwinds it if a step fails. Because each step commits independently, other transactions can see intermediate state before the saga finishes — the saga trades away the isolation guarantee of a classic ACID transaction (leaving you with roughly A-C-D) in exchange for not holding locks across service and network boundaries.

Compensations are not simple rollbacks

A database rollback discards uncommitted changes; a compensation runs after a commit, so it must produce a new committed state that undoes the effect. Two consequences follow. First, compensations are often semantic rather than exact: cancelling an already-charged payment issues a refund and leaves an audit trail rather than erasing history. Second, and this is the property every compensation must have, a compensation must be idempotent — a recovery process may re-issue it after a crash, and running it twice must have the same effect as running it once.

Idempotency is a blanket requirement. Commutativity is not — it is one specific countermeasure for a specific isolation problem, and we treat it that way in the pitfalls section below rather than demanding it of every compensation.

Two ways to coordinate the steps: orchestration and choreography

The saga structure says nothing about who decides which step runs next. There are two coordination styles.

In orchestration, a dedicated coordinator (an orchestrator, often implemented as a state machine) owns the workflow. It sends a command to each participant, waits for the reply, records progress, and decides the next command — including issuing compensations in reverse order when a step reports failure. Control flow lives in one place.

In choreography, there is no central coordinator. Each service reacts to events and publishes its own events; the next service is subscribed to those events and continues the saga. Control flow is emergent, distributed across the participants, with no single component that knows the whole plan.

A traced failure: order #7731

Concrete beats abstract. A customer places order #7731 for two units of SKU-88 at $60 each. The saga runs four local transactions across four services; payment is the step that fails.

T1, T2 and T3 each commit successfully. At T4 the card is declined, so nothing is charged and there is nothing to compensate for T4 itself. The saga now walks the backward path in reverse order of the successful steps:

1. C3 — Inventory releases the 2 reserved units of SKU-88.
2. C2 — Customer releases the $120 credit reservation.
3. C1 — Order sets #7731 to FAILED (a semantic undo: the row stays for audit, its state is terminal).

Reverse order matters: later steps may depend on earlier ones, so you unwind newest-first. When C1 commits, the system is back to a consistent state — no held credit, no held stock, and an order clearly marked failed.

The saga log and crash recovery

The orchestrator does not keep the saga's progress only in memory — if it did, a crash mid-saga would strand reserved credit and stock forever. Instead it writes a durable saga log: an append-only record of each step starting and finishing. Every forward and compensating transaction is bracketed by a BEGIN and a terminal record (END for success, ABORT for failure), so the log is self-describing and replayable. Here is the log for order #7731:

saga-7731  T1  Order.create        BEGIN
saga-7731  T1  Order.create        END    ok
saga-7731  T2  Customer.reserve    BEGIN
saga-7731  T2  Customer.reserve    END    ok
saga-7731  T3  Inventory.reserve   BEGIN
saga-7731  T3  Inventory.reserve   END    ok
saga-7731  T4  Payment.charge      BEGIN
saga-7731  T4  Payment.charge      ABORT  card_declined
saga-7731  C3  Inventory.release   BEGIN
saga-7731  C3  Inventory.release   END    ok
saga-7731  C2  Customer.release    BEGIN
saga-7731  C2  Customer.release    END    ok
saga-7731  C1  Order.markFailed    BEGIN
saga-7731  C1  Order.markFailed    END    ok

Now suppose the orchestrator crashes right after writing C3 … BEGIN but before C3 … END. On restart it reads the log, sees T4 aborted and C3 in flight with no terminal record, and simply re-runs C3. That is exactly why compensations must be idempotent: releasing the same reservation a second time is a no-op, so the replay is safe. The log turns an in-memory workflow into a recoverable one.

Pitfalls the architecture forces on you

Sagas remove distributed locks, but the problems locks solved do not vanish — they move into your application code.

Lack of isolation (lost updates, dirty reads)

Because each step commits immediately, a concurrent saga or query can read data that a later compensation will undo, or two sagas can overwrite each other. This is the isolation ('I') you gave up. Garcia-Molina and Salem, and later Richardson, catalogue specific countermeasures you apply per case — semantic locks (mark a record with a PENDING flag so others know it is in-flight), commutative updates (design the operation so order does not matter, e.g. debit/credit rather than set-balance), pessimistic ordering of steps, reread value before updating, version file (record operations and reorder them), and by value (route high-risk requests through a stricter path). Commutativity is one entry on this list — a targeted fix for concurrent-access anomalies — not a property demanded of every compensation.

Non-compensatable and pivot steps

Some actions cannot be undone: sending an email, calling a non-refundable external API, shipping a package. The standard structure is to order the saga as compensatable steps, then a single pivot step (the point of no return — once it commits the saga must go forward), followed by retriable steps that are guaranteed to eventually succeed. Put anything irreversible at or after the pivot.

Failed compensations

A compensation can itself fail (the service is down). There is no compensation-for-a-compensation, so the recovery machinery must retry it until it succeeds — which again requires idempotency — and escalate to alerting or manual intervention as a last resort.

Choreography-specific hazards

With no central coordinator, control flow is spread across event subscriptions. This invites event storms and hard-to-trace cascades, and cyclic dependencies where services indirectly trigger each other. And because messaging is typically at-least-once, handlers must cope with duplicate and out-of-order events — deduplicate via a processed-message table and keep handlers idempotent.

Choosing: saga vs 2PC, and orchestration vs choreography

Saga vs two-phase commit (2PC)

2PC gives you a genuine atomic distributed commit with full isolation, but at a price: a coordinator that is a single point of failure, participants that hold locks and block while awaiting the commit decision, a hard dependency on XA-style transaction support across every store, and poor availability under network partitions. It fits a small number of resources inside one latency and trust boundary, for short transactions where strong isolation is non-negotiable.

A saga holds no distributed locks, so it stays available and scales across heterogeneous and autonomous services — but it gives up isolation, only guarantees eventual consistency, and pushes real work onto you (writing compensations, ordering pivots, defending against anomalies). Choose a saga for long-lived business transactions spanning independent services where availability matters more than immediate global consistency.

Orchestration vs choreography

A common rule of thumb: start with choreography for simple sagas of two or three participants, and move to orchestration once the number of steps or the branching logic makes the implicit flow hard to follow.

Sources: Adapted and substantially expanded from the original lesson. Primary references: Chris Richardson, Microservices Patterns (Manning, 2018), Chapter 4, “Managing transactions with sagas”; and Hector Garcia-Molina and Kenneth Salem, “Sagas,” Proceedings of the ACM SIGMOD International Conference on Management of Data (1987), which introduces the compensation model and the concurrent-access countermeasures (semantic locks, commutative updates, reread value, version file) referenced above.