Every automated process has a decision to make about people: which steps still involve one, what they are actually reviewing, and what happens when they disagree with the system. Teams tend to answer this by reflex. Either everything gets reviewed, which removes most of the benefit, or nothing does, which works until the first expensive mistake.
The useful question is narrower than "should a human check this". It is which specific decisions carry enough cost when wrong to justify the delay of checking them.
Sort the decisions by what being wrong costs
Three categories cover most processes. Reversible and cheap: a misfiled document, a wrongly tagged ticket. Reversible and expensive: a mispriced quote that goes out and has to be withdrawn. Irreversible: a payment sent, a message to a customer, a record deleted.
Review belongs almost entirely in the third category and the expensive half of the second. Applying it to the first is where automation projects lose their return, because someone is now spending real time confirming outcomes that could simply be corrected if they were ever wrong.
Confidence is a poor trigger on its own
The common design routes low-confidence outputs to a person. It sounds principled and it has a flaw worth knowing about: model confidence correlates loosely with correctness, and it is often highest on exactly the inputs that are subtly out of distribution.
Confidence works better combined with something structural. Route on the value of the transaction, on whether the customer is new, on whether the case type is rare, on whether the output disagrees with an existing record. Those signals are independent of the model and they do not degrade when it is updated.
The reviews that catch real problems are almost never the ones the model flagged as uncertain. They are the ones a rule flagged because the amount was unusual.
Hannah Berg, Lead AI Engineer, Engineered With AI
Design against the rubber stamp
A review step where the reviewer approves almost everything is not a control. It is a delay that produces an audit trail. This happens whenever the interface makes approving easier than investigating, and it happens quickly: within weeks of a queue running at high accuracy, the reviewer is scanning rather than reading.
- Show the reviewer what the decision was based on, so they can check reasoning rather than just plausibility.
- Make rejecting as fast as approving, with a reason captured in one click.
- Track the approval rate. A queue running above roughly 98% approval is telling you either the threshold is wrong or the step should be removed.
- Sample a small share of what was auto-approved, so you learn about the errors that never reached a queue.
That last point is the one teams skip, and it is the only way to discover that the automated path is wrong in a category nobody thought to route.
The handover has to carry context
A person pulled into a process midway needs enough to decide without reconstructing the case themselves. In practice that means the input, the output, the reason it was routed to them, the relevant history, and the action they are expected to take.
When any of those are missing the reviewer opens three other systems to piece it together, and a step budgeted at thirty seconds takes four minutes. That is usually what turns a review queue from a control into a bottleneck, rather than the volume itself.
Decide what happens when they disagree
If a reviewer overrides the system, that override should go somewhere. Captured overrides are the highest-quality training and evaluation data a process produces, because each one is a specific case where a competent person judged the output wrong and said why.
Discarding them means solving the same problem repeatedly and never noticing the pattern.
Plan to reduce the queue over time
Review is expensive and it should shrink. Set the expectation at the start that categories graduate out of review once they have demonstrated accuracy over a meaningful sample, and that new or changed categories go back in.
Without that, the queue is permanent and the business concludes the automation never really delivered. With it, the cost curve bends the right way and the people stay pointed at the cases that genuinely need judgement. Our automation hub covers how this looks by process type.
Working out what still needs a person?
We will map your process and show you where review adds value and where it only adds delay.