An AI workflow should ask for review when the evidence needed for the next decision is missing, conflicting or outside its tested scope. Test that decision explicitly before rollout. A useful evaluation measures both whether the system completes appropriate work and whether it pauses at the right moment with enough context for someone to continue.
This Note proposes a practical evaluation framework for early-stage B2B teams. It is relevant to Benelux founders and Pakistani product teams serving international customers. It does not claim results from my own deployment or promise that a test set eliminates failures. The central question is simple: can the workflow recognise a case it should not complete alone?
Test a decision, not a confidence label
Start with one bounded workflow, such as preparing a draft answer to a supplier-document query. Write what evidence is required, what the system may do and which situations require review. “Ask when uncertain” leaves the decision too vague. “Pause when two current records disagree on the requested delivery date” creates a case that can be tested.
A model-generated confidence score is not, by itself, evidence that a decision is correct. Prefer observable conditions: a required source is missing, the account does not match, a document version is obsolete, or the request requires authority the workflow does not have. The product should express the reason for escalation in terms the reviewer can act on.
Anthropic’s January 2026 evaluation guide distinguishes a system’s final outcome from its transcript and discusses repeated trials. That supports testing what actually happened, not merely a convincing answer. The framework here is original operating guidance, not a benchmark result from that article.
Build cases where asking is the correct result
Create a small set of representative cases with expected outcomes written in advance. Include straightforward work, missing evidence, conflicting evidence and requests beyond the permitted scope. Use synthetic or appropriately authorised material. Keep a separate set of cases out of prompt development so the final check is not simply a rehearsal of examples used to tune the system.
For every case, define the required destination: complete the permitted draft, ask a specific clarification, or hand off to a named review queue. Include a case where extra questioning is unnecessary. A workflow that escalates everything can look cautious while giving the customer no useful automation. The test must reward appropriate completion as well as appropriate restraint.
Review the labels with someone responsible for the real process. If two knowledgeable people disagree about the expected action, clarify the business rule before scoring the model. Otherwise the evaluation may measure an unresolved internal disagreement rather than product quality.
Read a hypothetical scorecard honestly
Suppose a fictional test contains 100 cases: 60 should be completed automatically and 40 should be escalated. The workflow correctly completes 54 routine cases and unnecessarily escalates six. Of the 40 review cases, it escalates 35 and incorrectly completes five. These figures are illustrative, not observed product results.
The team can report 54 correct automatic completions, six unnecessary handoffs, 35 correct handoffs and five missed handoffs. Reporting only 89 correct decisions out of 100 conceals the operationally important difference between a delayed routine task and an unsupported action. Examine each missed handoff individually and decide whether its consequence prevents rollout.
There is no universal acceptable percentage. A draft that someone checks before sending has a different consequence from a workflow that changes a customer record. Keep review effort, error consequence and permitted action in the decision. Reducing the review queue is useful only if it does not hide cases that still require judgment.
Evaluate the handoff as a product feature
A handoff should state what the system tried to do, which evidence is missing or conflicting, what remains unchanged and what decision is required. Link to the relevant authorised record rather than copying unnecessary sensitive content. The reviewer should be able to continue without reconstructing the whole task from a long conversation.
Test ownership and timing too. Who sees the queue, what happens outside working hours and how does the customer know the request is pending? Record time to a usable resolution, not just time to the first notification. A technically correct escalation that nobody receives is an incomplete workflow.
Keep the rollout decision tied to evidence
The NIST AI RMF Measure playbook provides a broader reference for evaluating and documenting AI risks. It does not supply a pass mark for this hypothetical product. Choose release conditions for the actual task and record the reasons, limitations and responsible owner.
Before widening use, confirm that the system passes the agreed exception cases, the reviewer can resolve a sample of handoffs and an owner can narrow or pause automation. Repeat the relevant evaluation after changing the model, instructions, tools or business rules. Save the version and results so improvements are comparable rather than anecdotal.
The AI action-permissions Note addresses what an agent may do; this framework tests whether it recognises when to stop. The paid-pilot scorecard connects the evidence to a purchasing decision. Founders can share a concrete workflow, customer evidence and the next milestone. A credible automation story includes the work completed and the judgment deliberately left to people.