Human in the loop: designing automation that people trust
Not whether a person is involved but where. The gate pattern: AI proposes, rules route, a person handles the uncertain, all logged.
The short answer
Every useful automation involves people somewhere. The design question is where: before the action, checking each one; after it, sampling for quality; or only on exceptions the system flags. The right answer depends on what is at stake and how confident the system is, and the pattern that handles it is a gate. The AI proposes an outcome with a confidence level and its reasons. Rules you set route routine, confident cases straight through and everything else to a person who approves, edits or rejects in seconds. Every decision is logged, and the log is what lets you widen the straight-through lane over time. People trust automation they can see, correct and switch off.
Where the person sits
| Position | When it fits | Example |
|---|---|---|
| Before every action | High stakes, new system, low confidence | Every outgoing customer email is reviewed before sending |
| At the gate: exceptions only | Routine work with clear rules and a confidence signal | Invoices matching a purchase order post automatically; mismatches go to finance |
| After the action: sampling | Low stakes, high volume, mature system | Product descriptions published automatically; a person reviews a sample weekly |
| Not at all | Reversible, trivial, fully rule-based | Renaming files by a pattern |
Building the gate
- Define the proposal: what the AI produces, with a confidence signal and the evidence it used.
- Write the routing rules: stakes categories, confidence thresholds, patterns that always go to a person.
- Design the review screen: proposal, reasons, evidence, one-click approve, edit in place, reject with a reason.
- Start conservative: most cases to the person in the first weeks.
- Log everything: what was proposed, what was decided, by whom, what changed.
- Review the log weekly: which categories were approved unchanged; widen the straight-through lane for those.
- Keep the off switch obvious: anyone responsible can pause the automation in one action.
What trust looks like in practice
Staff can open any case and see what the system proposed, why, and who approved it. Corrections they make are visibly reflected in later behaviour. The straight-through lane widens over months, and everyone can see the log that justified it. When something goes wrong, the trail is complete and the fix is a rule change, not an argument. And the pause button is real. Automation designed this way is adopted; automation designed as a black box is worked around.
What this means for you
Put the person at the gate, not in front of every case and not out of the picture. Let the AI propose with reasons, route by stakes and confidence, log every decision, and widen automation from the evidence. Keep the machinery visible and the off switch close. Automation built this way saves the time you hoped for and keeps the accountability you cannot afford to lose.
Frequently asked questions
Does a person in the loop not defeat the point of automating?
Only if the person does the same work as before. In the gate pattern the person reviews a prepared proposal with reasons, in seconds, for the minority of cases that need judgement. The routine majority passes without them. The time saved is real; what is preserved is accountability on the cases that matter.
How do we decide what goes straight through?
By stakes and confidence, from the log. Low stakes and high confidence: straight through. Money, commitments, people's data or an unusual pattern: a person. Begin conservatively, review weekly what the person approved without changes, and move those categories into the straight-through lane when the evidence supports it.
What happens when the person just approves everything?
That is rubber-stamping, and it is a design failure: too many cases routed to review, or reviews presented without the information needed to judge. Fix it by narrowing what reaches the person to genuinely uncertain cases, showing the reasons and the evidence, and sampling approvals for quality. The person's attention is the scarce resource; spend it where it matters.