Batch Processing and Multi-Pass Review

The previous chapters made a single call reliable: precise, structured, validated. This one changes the frame from one request to many, and from producing output to judging it. Two skills sit here, and both are the same move at heart: matching the shape of the work to the shape of the workload. Run a thousand requests the wrong way and you overpay or block your pipeline; review a large output the wrong way and you miss the bugs that matter.

What batch processing is for

By default you call the model synchronously: send a request, wait, get an answer, one at a time. That is the right shape when a human or a pipeline is waiting on the result now. It is the wrong shape when you have a large pile of independent requests and nobody is waiting on any particular one: ten thousand documents to classify overnight, a weekly audit, a nightly test-generation run. Firing those synchronously means paying full price and babysitting throughput for work whose deadline is “by morning.”

Batch processing is the shape that fits that second case. You hand the model the whole pile as one asynchronous job and collect the results later. In exchange for giving up “now,” you get a much cheaper “later.” The Message Batches API is that mechanism, and its trade-offs are the exam content — anthropic 0.120.0 exposes client.messages.batches (create, retrieve, results, list, cancel):

• 50% cost savings versus synchronous calls.
• Up to a 24-hour processing window, with no guaranteed latency SLA: you submit, and results come back sometime within that window.
• custom_id on each request correlates it with its response, since results don’t come back in order.

When batch fits, and when it doesn’t

Those trade-offs make the fit obvious: batch is for non-blocking, latency-tolerant workloads — overnight reports, weekly audits, nightly test generation — where “sometime in the next several hours” is fine. It is wrong for blocking workflows: a pre-merge CI check can’t wait up to 24 hours for a review, so that must use the synchronous API. The exam’s clean split: synchronous for blocking (pre-merge checks), batch for overnight/weekly analysis.

One hard constraint that’s a favorite exam trap: the Batches API does not support multi-turn tool calling within a single request. It can’t execute a tool mid-request and feed the result back — there’s no agentic loop inside a batch item. So batch is for single-shot work (extract these fields, classify this document), not for agentic tasks that need tools. If an item proposes running a tool-using agent over a batch, that’s the trap; batch items are one-and-done.

Two operational skills round it out. SLA arithmetic: if you need a 30-hour end-to-end SLA and batch processing takes up to 24 hours, you submit on a window tighter than the remaining 6 — e.g. every 4 hours. A job submitted at the worst moment still finishes in time. And failure handling: resubmit only the failed items, identified by custom_id, with appropriate fixes (chunking a document that exceeded the context limit) rather than re-running the whole batch. Refine your prompt on a small sample before committing a large volume, so you’re not paying to discover a prompt bug across ten thousand documents.

Why you can’t review your own work

The second skill is review, and it starts from a question worth asking plainly: who should check the model’s output? The obvious answer — the model that produced it — is the wrong one. A model reviewing its own output in the same session is a weak reviewer. The generating session retains its own reasoning: the assumptions it made, the design it committed to. So it’s less likely to question those decisions. It already “knows” why the code is right, so it rationalizes past the flaws instead of catching them. This isn’t fixed by asking it to self-review harder, nor by extended thinking; the problem is the shared context, not the effort.

The fix is an independent review instance: a fresh Claude with no prior reasoning context, given only the artifact to review. Without a stake in the original decisions, it evaluates the output on its merits and catches subtle issues the author-session waved past. This is the same independent-reviewer principle the CI/CD chapter reached from the other direction: the value of the review is precisely that it doesn’t share the generator’s context. When an exam item offers “have the model review its own work” versus “use a second independent instance,” the second instance wins.

Multi-pass review for large work

A fresh reviewer solves the bias problem but not the scale problem, and large reviews fail a second way: attention dilution. Ask one instance to review a 40-file change in one pass and it spreads its attention thin, misses local issues, and produces contradictory findings across files. The fix is multiple focused passes:

• Per-file local passes — review each file on its own for local issues (this function’s bug, this file’s style), where the model can attend fully to a small scope.
• Cross-file integration passes — separately, review the interactions that no single-file pass can see: data flow between modules, an interface change that ripples across files.

Splitting the work this way beats one big pass because each pass has a narrow, coherent scope the model can actually hold. Local correctness where the file is the unit; integration correctness where the data flow is the unit. It’s decomposition applied to review — complete coverage from focused pieces rather than diluted coverage from one overloaded pass.

A supporting technique reaches into Domain 5: run verification passes where the model self-reports confidence alongside each finding. You can then route review attention by confidence — sending the low-confidence findings to a human and trusting the high-confidence ones. (The confidence must be calibrated to be useful, which is a Domain 5 concern; here the point is that a confidence signal enables routing.)

Final thoughts

Match the architecture to the workload. The Batches API trades latency for 50% savings and fits overnight, latency-tolerant, single-shot work. Never blocking checks, and never tool-using agents, since a batch item has no tool loop. For review, an independent instance with no shared reasoning beats a self-review that rationalizes its own decisions, and per-file plus cross-file passes beat one attention-diluted pass over a large change. Both tasks are the same lesson: reliability at scale comes from choosing the right shape. Asynchronous where you can tolerate the wait; decomposed and independent where a single overloaded pass would miss things. That completes Domain 4.

Next: Arc 5 opens with context management — keeping the critical facts alive across long conversations and large codebases.