Discovery and the Art of the Tradeoff Conversation

The cheapest failure to fix is the one you catch before you build. The most expensive is the one where you build the wrong thing beautifully, ship it, and discover in production that it solves a problem nobody had. Everything between those two outcomes is decided in a phase most engineers rush through: discovery. This chapter is about doing it deliberately. It is also about the second half of the same job: communicating what you found and what you decided to the people who signed the check.

What discovery actually is

Discovery is the work of turning a stakeholder’s stated want into a specification you can design against. A stated want is compressed and optimistic: “we need an AI assistant that handles customer support.” That sentence contains a proposed solution, an implied budget, and a hope. It contains almost none of what an architect needs to draw a single box. Discovery is how you pull the real requirements out of it.

Those requirements come in two families, and keeping them separate is half the discipline.

Functional requirements are what the system must do. The assistant must answer questions about orders. It must look up an order by number or by customer email. It must process a refund when the customer is eligible. It must escalate to a human when it cannot resolve the issue. Each of these is a capability you can point at and test.

Non-functional requirements are the constraints the system must satisfy while doing it. They are where discovery earns its keep, because stakeholders almost never volunteer them. Four of them recur. The SLAs: a p95 response under three seconds, 99.9% availability. The cost budget: a ceiling on dollars per resolved ticket. The compliance boundary: customer PII stays in-region, and refunds above a threshold get human review. And the success metric: what number, measured how, tells everyone this worked. A functional requirement says the system answers refund questions. A non-functional requirement says it does so in under two seconds, for less than four cents, without ever leaking one customer’s data into another’s session. You cannot design an architecture from the first list alone. The second list is what forces every hard choice.

Why it is the highest-leverage phase

Discovery is the highest-leverage work an architect does, and the reason is arithmetic. A requirement missed in discovery is nearly free to add now and ruinous to add later. Suppose nobody surfaces the constraint that refunds over $500 need a manager’s approval. You will design an agent that processes them autonomously, build it, and test it against a happy path that never trips the rule. Then you meet the constraint for the first time in a compliance review after launch. The redesign touches the tool layer, the guardrails, the human-in-the-loop path, and the evals. The same fact, written down in week one, is a single line in the spec.

Building the wrong thing well is the most expensive failure mode in software, and it is almost always a discovery failure rather than an engineering one. The team that ships a flawless ticket-deflection bot when the business needed handle-time reduction did nothing wrong at the keyboard. They answered a question nobody had asked them to verify. An hour spent confirming which problem you are solving is worth a month spent solving it.

When discovery is not the bottleneck

Discovery deserves its status, but treat it as judgment rather than ceremony. Two failure modes sit on either side of it.

The first is skipping it, covered above. The second, less discussed, is drowning in it: an endless requirements-gathering phase that produces a hundred-page document and no shipped system, where the org uses “we’re still in discovery” as a way to avoid committing to anything. Discovery is done when you can state the problem, the functional and non-functional requirements, and the single success metric clearly enough to design against. It is not done only once every conceivable edge case has been catalogued. Some requirements can only be learned from a running system meeting real users. For those, the right move is to ship a scoped first version and discover the rest from production. Knowing which requirements you must nail before building, and which you can safely learn afterward, is itself an architectural call.

The landscape of what to elicit

A structured discovery covers a predictable set of territories. Walk them explicitly so none is left as an assumption.

• Users and their context. Who uses this, how many, how sophisticated, in what channel? A support assistant for enterprise account managers is a different system from one for anonymous shoppers: different tone, different auth, different tolerance for a wrong answer.
• Data. What does the system read from and write to? Where does it live, how fresh is it, who owns it, and is it clean? The stale-knowledge-base problem is discovered here or not at all. For a Claude system this is also where you learn what the RAG corpus is and whether it exists yet.
• Latency and cost budgets. What is the response-time SLA, and what is the ceiling on cost per interaction? These two numbers, more than any other, decide model tier, caching strategy, and whether you can afford a multi-agent design. Get them as ranges if you cannot get them as points.
• Compliance and risk constraints. What regulation applies (GDPR, HIPAA, sector rules)? What data is sensitive, what must be auditable, what actions are irreversible and therefore need a gate? An architect elicits what the architecture must provide here, which is the decision-level view of compliance, rather than posing as a lawyer.
• Success criteria. The single most-skipped item. What number, measured how and by whom, declares this a success? “Better support” is not a criterion. “Median resolution time down 40%, and 60% of tickets resolved without a human, measured monthly, weighted to enterprise accounts” is one you can design toward and later prove.

Elicit the constraints before you commit to a shape. Every constraint you learn late is a redesign. Every one you learn early is a line in a document.

Communicating decisions and tradeoffs

Discovery produces requirements. The second half of the architect’s stakeholder job is producing decisions, and then making a non-technical sponsor able to own them. This is the part engineers most often fumble, because the instinct is to explain the mechanism when the sponsor needs to weigh the outcome.

The core move is to frame every hard choice as a tradeoff the business owner can actually make. Almost every architectural decision in a Claude system reduces to some balance of three axes: quality, cost, and latency. You cannot maximize all three, and pretending you can is how you lose a stakeholder’s trust the first time reality contradicts you. Your job is to translate the engineering choice into those three currencies and let the person who owns the budget choose the balance.

That means presenting options, not a verdict. “We should use the mid-tier model” invites either blind agreement or an argument you are not equipped to win. Offer a small menu instead. Option A is faster and cheaper, and will get roughly one answer in twenty subtly wrong. Option B costs three times as much per ticket and takes a second longer, and cuts that error rate by half. Each option is named in cost per ticket, in seconds, and in a quality number the sponsor can feel. Now the VP is making a business decision with real inputs, which is exactly the decision that belongs to them and not to you.

Two disciplines make this land. Manage expectations honestly, especially about SLAs and about the fact that a Claude system is probabilistic. A sponsor who is told “it will be right about 95% of the time, and here is what happens on the other 5%” and then sees a wrong answer is watching the system behave as promised. A sponsor who was told “it works” is watching it break. Under-promise on the axis you are least sure of. And use their language, not yours: cost per resolved ticket, not tokens; seconds a customer waits, not p95 latency; “one answer in twenty needs correcting,” not an eval F1 score. The translation is the work.

The bookshop platform, in discovery

Run this on the bookshop support platform before a single box is drawn.

Six things surface. The users: enterprise account managers and end shoppers, two audiences with different auth and different stakes on a wrong answer. The data: a policy corpus, catalog, and help articles for RAG, plus the live orders database, and crucially whether the help articles are current. The latency budget: the assistant must feel instant, so a low-single-digit-second p95. The cost ceiling: deflection only saves money if a resolved ticket costs less than a human-handled one, which sets a hard per-interaction cap. The compliance boundary: customer PII under GDPR, refunds above a threshold gated to a human. The success metric: resolution time and no-human resolution rate, weighted to the high-value accounts that churn. None of that was in “build an AI support assistant,” and all of it drives the design.

Then comes the tradeoff conversation, and the sharpest one is model tier. You do not walk into the VP’s office and say “we’ll use Haiku for retrieval and Sonnet for the hard cases.” You say: here are three ways to build this. The cheap build runs a small model throughout, costs roughly two cents a ticket, and gets about one answer in fifteen wrong, which a human then has to catch. The premium build runs a large model throughout, costs closer to nine cents, and cuts that error rate by more than half, but a large share of tickets are simple lookups that do not need it. The recommended build routes by difficulty. The small model handles the easy majority and escalates the hard minority to the large one. That lands near three cents a ticket, at close to the premium quality on the cases that matter. Each option is a dot on the quality-cost-latency space, priced in the VP’s currency. You have a recommendation, and you say so, but the VP is choosing with real numbers rather than ratifying a decision they cannot see into. That is what it means to communicate an architecture.

What the exam is really checking

Domain 6 items in this area put you in the room with a stakeholder. A sponsor states a want, or asks why the system costs what it does, or pushes for a guarantee the technology cannot make. The trap answers skip discovery: they design from the stated solution instead of eliciting the real requirements. Or they communicate badly, handing a non-technical owner a technical verdict, or promising a certainty a probabilistic system cannot honor. The credited answers do what this chapter did: separate functional from non-functional requirements, surface the constraints before committing to a shape, and frame the hard choice as a quality-cost-latency tradeoff the business owner can weigh in their own terms. Discovery is the leverage, and the tradeoff conversation is how the leverage reaches the person who decides.

Next: documenting an architecture and shepherding it through its lifecycle — the decision record that survives the handoff, and why a Claude system is never done.