Precision Prompting and Few-Shot Examples

A prompt is an instruction, and the quality of what a model gives back is bounded by the quality of that instruction. This is the part of the craft the exam calls precision: writing prompts that produce output you can actually rely on, instead of a noisy stream you have to second-guess. The two chapters before this one configured Claude Code — its files, its commands, its skills. This one steps back to the prompt itself, because a well-configured agent driven by a vague prompt is still a vague agent.

What “precision” means, and why it is hard

Precision here is not about writing more, or writing more politely. It is about closing the gap between what you mean and what you said. A model does not share your context or your intuitions; it acts on the words in front of it. When those words are open to interpretation, the model interprets. And it will interpret differently across runs, across inputs, and often not the way you intended. That variance is the enemy. A code reviewer that flags real bugs on Monday and stylistic nitpicks on Tuesday is not a reviewer you trust, however smart it is on any single run.

The tempting fix is the wrong one: you cannot buy precision by asking the model to be careful. “Be conservative,” “use your best judgment,” “only report high-confidence findings” — these read like precision controls and deliver none. The model has no calibrated sense of its own confidence to threshold against. Precision comes from telling the model exactly what to do, not from asking it to feel more strongly about doing it well. Two techniques carry that load: explicit criteria and few-shot examples. Their effectiveness is a claim about model behavior, so the concrete results below were checked; the principles are the exam content.

Explicit criteria beat vague instructions

The foundational lesson: specific, categorical criteria outperform general instructions. Telling a code reviewer to “check that comments are accurate” is vague. The model has to invent what “accurate” means, and it will flag stylistic quibbles alongside real bugs. Telling it to “flag a comment only when the claimed behavior contradicts the actual code behavior” is a precise, checkable criterion, and it produces precise findings.

The corollary: generic hedges don’t improve precision. “Be conservative,” “only report high-confidence findings,” “use your best judgment” — these feel like they’d cut false positives, and they don’t. The model has no calibrated notion of its own confidence to threshold against, so a confidence instruction is noise. What actually reduces false positives is specific categorical criteria that define which issues to report (bugs, security vulnerabilities) versus skip (minor style, local patterns). You get precision by naming the categories, not by asking the model to be careful.

Precision matters because of a trust dynamic: a high-false-positive category poisons the accurate ones. If a reviewer’s “style” findings are usually wrong, developers stop trusting all of its findings, including the correct security ones. The noise trains them to dismiss the tool. So a category that fires too many false positives isn’t just locally bad; it undermines the whole system’s credibility.

That gives a concrete remediation: temporarily disable a high-false-positive category while you improve its prompt, to restore trust in the categories that work. Turning off the noisy check isn’t giving up — it’s protecting the signal. And for classification tasks, define explicit severity criteria with a concrete code example for each level. Then “high” versus “medium” severity is anchored to examples rather than left to the model’s shifting judgment.

Few-shot: teaching judgment, not matching

Explicit criteria take you a long way, but they hit a wall: you cannot spell out every case. Where the criteria run out, the model improvises — usually in the format of its answer, or in how it handles a genuinely ambiguous input. When that happens, few-shot examples are the most effective next technique. You show the model 2–4 worked examples of exactly what you want, and it patterns its output on them.

The framing of few-shot is subtler than “show examples.” Few-shot examples don’t just make the model match the cases you showed — they let it generalize its judgment to novel patterns. A handful of well-chosen examples teaches the model the principle behind your choices, which it then applies to inputs you never demonstrated. That’s why few-shot beats an exhaustive rule list: you can’t enumerate every case, but a few examples that reveal the underlying judgment cover the ones you didn’t.

The highest-value examples are the ambiguous ones, shown with their reasoning:

Example — ambiguous tool selection:
Request: "What did we spend on cloud last quarter?"
Reasoning: This needs the billing tool, not the general search tool, because it asks
for a specific financial figure over a defined period. Search would return docs
*about* cloud spend; billing returns the number.
Chosen: get_billing(category="cloud", period="last_quarter")

Showing why one action was chosen over a plausible alternative teaches the model to make the same distinction on a new ambiguous request. A rule (“use billing for financial questions”) can’t do that when the phrasing is unexpected. This pays off in several places: demonstrating a specific output format (location, issue, severity, suggested fix) for consistency; distinguishing acceptable code patterns from genuine issues to cut false positives while still generalizing; and showing correct extraction from varied document structures (inline citations vs. bibliographies) so the model handles formats it wasn’t explicitly shown. (On a ticket-classification task with a strict output format, zero-shot produced 0 of 4 responses in the required format, while a two-example few-shot prompt produced 4 of 4 — a sharp jump in consistency. That it also reduces hallucination is documented model behavior.)

Precision plus generalization

The two techniques compose. Explicit criteria set the boundary: what counts as a finding, what doesn’t, at what severity. Few-shot examples teach the judgment to apply that boundary to cases you didn’t spell out, especially the ambiguous ones. Criteria alone can be rigid and miss novel-but-valid cases; examples alone can be imitated too literally without the underlying rule. Together, they give you output that’s both precise and able to generalize: a clear criterion, plus a few examples showing the reasoning at the boundary. When an exam item’s prompt is producing inconsistent or noisy results, the fix is rarely “add ‘be careful’”; it’s a sharper criterion, a few reasoned examples, or both.

Final thoughts

Precision comes from specific categorical criteria, not from confidence hedges the model can’t calibrate. It matters because a noisy category poisons trust in the accurate ones, which is why temporarily disabling a high-false-positive check is a legitimate move. Few-shot examples teach the model to generalize its judgment to novel patterns, and their most valuable form shows the reasoning behind an ambiguous choice. The pairing — explicit criteria for the boundary, few-shot for the judgment at it — is the reliable way to turn an inconsistent prompt into a trustworthy one. Next we make the output itself reliable, not just the reasoning behind it.

Next: structured output and validation loops — guaranteeing schema-compliant output with tool use, and the retry loops that fix what schemas can’t.