Publish What the Database Committed

The order commits, then the process stops before sending its notification. If the only reminder lived in the running flow, nothing remains to tell the warehouse about the accepted order. Sending first has the opposite problem: the receiver can act on an order that later fails to commit.

An outbox stores notification intent in the same database transaction as the order, so the two records live or die together. A separate dispatcher then reads that intent and attempts delivery. The dispatcher still faces uncertain network outcomes — which is why the receiving side, not the sender, is where a repeated event identity has to be recognized.

Open this checkpoint in ACB

Stop the previous local application, then use File → Open Folder to open book/checkpoints/19-outbox. Open src/main/mule/app.xml and use Flow List to select the named flow for each example. Start the controlled dependency in a separate terminal from the companion root with python3 book/stubs/server.py. Choose Run and Debug → Run Mule Application and wait for deployment. Save canvas edits and use Save and Hot-deploy to Local Runtime before repeating a request.

Run python3 book/run.py verify 19 from the companion root to exercise this running checkpoint with its synthetic fixtures. The verifier supplies requests and checks results; it does not start the ACB application. Keep the editor on this checkpoint while reading a failure so an old deployment cannot supply a misleading answer.

Commit the intent with the order

Checkpoint 19 adds outbox(event_id, body, sent) to the file-backed teaching database. The event identity combines tenant, order and event version: retailer-a:A-1001:accepted-v1. Its body contains the recorded order and type: "order.accepted".

Example 050 — Commit notification intent with acceptance

The complete source is in checkpoints/19-outbox/src/main/mule/app.xml.

Choose Flow List → commit-order in checkpoint 19. Compare its transactional Try with chapter 18. The request-result and accepted-order Inserts remain; a third Insert follows the failure-injection Choice. Select that third operation and inspect:

Field	Value
Connection Config	Orders_DB
Target Variable	outboxResult
Transactional Action	ALWAYS_JOIN
SQL	INSERT INTO outbox(event_id,body,sent) VALUES (:id,:body,FALSE)

Its Input Parameters expression builds the stable event identity and JSON body:

{
  id: vars.tenant ++ ":" ++ vars.order.orderId ++ ":accepted-v1",
  body: write({
    eventId: vars.tenant ++ ":" ++ vars.order.orderId ++ ":accepted-v1",
    "type": "order.accepted",
    order: vars.order
  }, "application/json")
}

Keep all three Inserts under the same Try, whose transaction settings are ALWAYS_BEGIN and LOCAL. Its On Error Propagate still matches ANY. Moving the outbox Insert outside that scope changes the acceptance guarantee.

The third insert joins the same transaction as the request result and order. After a successful acceptance the state endpoint reports one order, one request result and one pending event. Before commit, the injected failure must leave none of those records. This is a database atomicity claim — it does not include the receiver’s database in the transaction.

The stable event ID is distinct from a request correlation ID. A delivery retry is a new execution that still carries the same event identity. If every attempt invented a fresh ID, the receiver could not recognize the repeated effect.

Dispatcher sends the event before updating its outbox row.

Publish a bounded page

The dispatcher’s job is to turn recorded intent into a delivered event without ever inventing one. It reads a bounded page of unsent rows, sends each one, and marks a row only after the receiver has answered.

Example 051 — Dispatch pending order events

The complete source is in checkpoints/19-outbox/src/main/mule/app.xml.

Choose Flow List → dispatch-pending. Select the first Select, using Orders_DB, and inspect its bounded query:

SELECT event_id AS "eventId", body AS "body"
FROM outbox WHERE sent = FALSE ORDER BY event_id LIMIT 10

Expand For Each. Set Variable saves the current row as pending. Select Request: Method POST, Path /events, connection Dependency_HTTP, and Target Variable receipt. Its Body expression is:

output application/json
---
read(vars.pending.body as String, 'application/json')

The Headers expression is {'Content-Type': 'application/json'}. The following Update uses Orders_DB, Target Variable marked, SQL UPDATE outbox SET sent = TRUE WHERE event_id = :id, and Input Parameters {id: vars.pending.eventId}.

The Transform Message after For Each uses JSON output and body {attempted: sizeOf(payload)}. Keep Update after Request: marking first would forget an event that was never delivered.

The Select reads at most ten unsent records in deterministic identity order. For Each preserves the original selected collection, which lets the final transform report its size. Before publishing, the flow saves the current row in vars.pending; the HTTP result goes to a target so the later Update retains the event identity.

The database flag changes only after a successful response. If the call fails, the row remains pending — which is the reason for writing the intent down in the first place. The local /lab/dispatch endpoint maps the selected dependency failures to status 503 with PUBLICATION_UNCERTAIN.

This dispatcher has one owner. Running two independent copies can select the same pending row. The receiver’s deduplication limits repeated effects in this lab — but it does not make competing dispatchers efficient, and it does not establish ordered delivery. A multi-owner design needs a tested claim/lease protocol and recovery of abandoned claims. Adding CloudHub replicas does not implement that protocol.

Let the receiver show the ambiguity

The supplied Python service on port 18882 stores each event in SQLite with an event-ID primary key. An identical repeat returns the existing acceptance; different content under the same identity conflicts. Its durable row is the demonstration effect. It does not represent stock reservation, payment or warehouse fulfillment.

The interesting failure is the one where the effect happens and the answer does not. POST /control with loseResponseNext set to true; the next /events call commits its row, then deliberately returns 503. From the dispatcher’s side, publication remains uncertain — from the receiver’s state endpoint, one effect exists.

Run /lab/dispatch again. The same event is sent, the receiver recognizes it, and the dispatcher marks the outbox row sent. The expected local state is one receiver effect after two delivery attempts, with zero pending outbox rows. A crash between publishing and marking the row has the same shape — the event is published again, and only the receiver’s identity check keeps the effect single.

A successful HTTP status needs a contract too. Here it means the receiver durably recorded the event. If a real endpoint returns success before its durable handoff, that response cannot justify marking publication complete under the same promise.

Recovery needs identities and ownership

An outbox row marked sent is still evidence, so retain those records, or an audit trail, for the period required to explain accepted orders. An outbox cleanup job must distinguish acknowledged publication from abandoned work, and its retention cannot undermine receiver deduplication — if the receiver forgets an event while an old sender can still replay it, the effect may recur.

Ordering is another contract. Event IDs sort deterministically here, but lexical order is not a business sequence. Multiple updates for the same order need a version and a receiver rule for stale or out-of-order events. The one-time acceptance example does not prove that update protocol.

Monitor the oldest pending age as well as the count. Ten pending events created a second ago may be healthy — one waiting for a day can represent a broken customer promise. The recorded intent gives operations something concrete to investigate even if the original process is gone.

Try it

1. Stop before commit. Which records may remain after the injected transactional failure?

Show answer

None of the request result, accepted order or outbox intent from that transaction. Query the state independently after the failing request.

2. Stop after the receiver commits. Why must the next delivery keep the same event ID?

Show answer

The receiver needs that identity to return its prior acceptance without recording the effect again. A fresh ID would describe a different event to its deduplication boundary.

3. Add a second dispatcher. What must change before claiming exclusive publication ownership?

Show answer

Exclusive publication needs a tested claim or lease mechanism with conditional acquisition, expiry/recovery and owner checks. Shared access to the outbox table is not itself exclusive ownership. Receiver idempotency stays even with claims, because acknowledgements can still be lost.

An outbox separates acceptance from publication without forgetting either. A queue can add buffering and redelivery between publisher and consumer, and it still needs the same agreement about what an acknowledgement means: the queue handles delivery after publication, while the outbox handles getting there.

Next: Give Unfinished Work an Owner