The Saga Pattern: Choreography vs Orchestration, Through Amazon's Checkout Flow

A customer's card has already been charged. Then Promotions Service comes back: the coupon was redeemed twice, in a race, and it's already gone. In a monolith this never happens — one transaction, commit or rollback, done. Split across five services, "undo the last three things that already succeeded in their own databases" is a real problem with no free answer.

Why Two-Phase Commit Doesn't Survive This

In a monolith, checkout is one ACID transaction: reserve inventory, charge the card, create the order, deduct the coupon balance — commit or rollback, done. Split that into services (Order, Inventory, Payment, Promotions, Fulfillment, Notification) and you lose the ability to wrap them in a single database transaction. Two-phase commit technically solves this but doesn't survive at internet scale: it holds locks across every participant for the duration of the transaction, tolerates network partitions badly, and turns every service into a potential point of blocking for every other one.

The Saga pattern replaces one big ACID transaction with a sequence of local transactions, each committed independently in its own service, with a compensating transaction defined for every step that might need to be undone if a later step fails. There's no distributed lock. Instead of "all or nothing, atomically," a saga guarantees "all steps eventually complete, or all completed steps get compensated" — eventual consistency with backward recovery.

Two coordination styles do this:

• Choreography — each service publishes events; others subscribe and react. No central brain.
• Orchestration — a central coordinator tells each service what to do next and reacts to their responses.

Here's the six-step checkout flow both styles have to handle, with the compensating action for each step:

Notification is a good example of a step that isn't compensated the same way as the others — some steps are pivot points beyond which you don't roll back, you only roll forward. More on that once you've seen it happen below.

Choreography: Let the Services Talk

Each service owns its slice of the workflow and reacts to domain events on a broker (Kafka, SNS/SQS, EventBridge). No orchestrator exists.

If PaymentFailed publishes instead, InventoryService (subscribed to it) releases the reservation, and OrderService (also subscribed) cancels the order. Every service knows which failure events to listen for and what to compensate — there's no single place giving orders.

Orchestration: Put One Brain in Charge

A central Saga Orchestrator — a dedicated service or a workflow engine — owns the sequence, calls each participant directly, waits for the result, and decides what happens next, including firing compensations in reverse order on failure.

If PaymentService.charge() fails, the orchestrator — knowing the saga's defined step sequence — invokes compensations in reverse: InventoryService.releaseStock(), then OrderService.cancelOrder(). It marks the saga FAILED, the order CANCELLED, and can notify the customer with an accurate reason immediately, because it's the one place that knows exactly what happened.

Run It Yourself: Inject a Failure, Watch It Recover

Both diagrams above show the happy path. The interesting behavior only shows up when something fails partway through — and specifically, whether the failure happens before or after the step marked pivot below changes what "recovery" even means. Pick a mode, pick where it breaks, and run it.

The Pivot Transaction: Where "Undo" Stops Working

Notice what happened when you injected a failure at Promotions, Fulfillment, or Notification versus at Order, Inventory, or Payment. Payment is the pivot transaction: the step after which you no longer compensate backward, you only retry forward until it succeeds.

Once money has actually moved, you don't "compensate" by silently cancelling the order — the card was really charged. You either complete the order (retrying whatever failed downstream, as the simulator just showed) or you issue an explicit, audited refund, which is its own real transaction, not a saga rollback. Steps before the pivot (order creation, inventory reservation) are freely compensatable, because nothing external and irreversible has happened yet. Steps after it are retried, not rolled back, because rolling back would mean pretending money that moved didn't.

Every saga has a pivot somewhere, even if the design doc never uses the word — it's whichever step is the first one that's expensive, external, or embarrassing to undo. Naming it explicitly is what turns "we'll just add a compensating transaction for everything" from a slogan into an actual design decision.

Choosing Between Them

In practice, most large e-commerce systems use a hybrid: orchestration for the core "must complete reliably, must be auditable" checkout backbone (order → inventory → payment), and choreography/pub-sub fan-out for downstream, non-critical, fire-and-forget concerns (analytics, recommendation refresh, loyalty points, marketing triggers) that don't need to participate in compensation logic at all. The backbone gets a coordinator; everything that doesn't need one doesn't get one.

What's Actually Running This in Production

If you're already AWS-native, Step Functions is the lowest-friction orchestration option since it composes directly with existing SQS/Lambda infrastructure. Temporal earns its extra operational weight once workflows get long-running, need human-approval steps, or you want strong local-dev testability of the workflow logic itself rather than YAML/ASL.

The Concerns Neither Style Solves For You

• Idempotency — every step, including every compensation, must be safe to retry. Use idempotency keys per saga-instance + step.
• Ordering guarantees — choreography needs partitioned, ordered topics (e.g. Kafka partitioned by order ID) so one saga instance's events aren't processed out of order.
• Timeouts — define a max wait per step; a stuck downstream service shouldn't hang a saga indefinitely.
• Observability — a correlation/saga ID on every log line and event, so one order's full journey is traceable across services regardless of which style is running it.
• Compensation is not rollback — a compensating transaction is its own real business operation (a refund is a real, audited transaction) and needs its own idempotency and auditing, not a magic undo.

Further Reading

• Chris Richardson, Microservices Patterns — the standard reference for the saga pattern, choreography vs orchestration, and the pivot transaction concept (microservices.io/patterns/data/saga.html)
• Hector Garcia-Molina & Kenneth Salem, "Sagas" (1987) — the original database-systems paper the pattern is named after
• AWS Prescriptive Guidance: Saga pattern — a Step Functions-based reference implementation
• Temporal documentation: Saga pattern with Temporal
• Camunda: Saga Pattern in Microservices
• Martin Fowler, "Event-Driven Architecture" — background on choreography-style tradeoffs