The pitch for agentic AI is intoxicating: point a system at a goal and let it plan, call tools and act, all by itself. The demos are dazzling. The production incidents are educational. Somewhere between the two sits the question that decides whether an agent is an asset or a liability — not "can it act on its own?" but "where should it stop and ask a human?"
Full autonomy is almost never the right target. The best agentic systems we operate are not the most autonomous ones; they are the ones that automate the ninety-five percent that is safe and boring, and route the five percent that is risky or ambiguous to a person — cleanly, with context, at exactly the right moment. This is a framework for finding that line and designing the handoff around it.
Autonomy is a spectrum, not a switch
Treating "human-in-the-loop" as on-or-off is the first mistake. There is a ladder of oversight, and different actions belong on different rungs:
- Fully autonomous. The agent acts and reports afterward. Right for read-only, low-stakes, reversible work.
- Notify. The agent acts but surfaces what it did prominently, so a human can catch and undo it quickly.
- Approve before acting. The agent prepares the action and waits for an explicit human confirmation before it executes.
- Human does it, AI assists. The agent drafts, recommends and pre-fills, but a person performs the final act. Right for the highest-stakes decisions.
A mature agent uses all four rungs at once, choosing per action rather than adopting one posture for everything.
Three questions that place the line
For any action an agent can take, we ask three questions. The answers place it on the ladder.
1. Is it reversible?
Reversibility is the single most important axis. Reading a database, drafting an email, proposing a plan — all trivially undoable, all safe to automate. Sending money, deleting records, emailing a customer, merging to production — hard or impossible to take back. The cost of a wrong reversible action is a few seconds; the cost of a wrong irreversible one can be a customer, a fine, or a headline.
Automate the reversible freely. Gate the irreversible deliberately. If you remember one line from this piece, make it that one.
2. What is the blast radius?
An action that touches one record for one user is very different from one that touches a hundred thousand. Scale changes the calculus even when reversibility does not. An agent updating a single support ticket can run free; the same agent about to update every ticket matching a filter should stop and show a human the count and a sample before it proceeds. Bulk operations deserve a pause on principle.
3. How confident is the model, really?
Confidence should modulate autonomy dynamically. A high-confidence, well-grounded decision on a routine case can flow through; a low-confidence or novel one should escalate — even if the same action would normally be automatic. The trap is trusting a model's self-reported confidence, which is often miscalibrated. Ground the estimate in real signals: agreement between multiple attempts, the strength of retrieved evidence, whether the input resembles cases the system has handled before, and whether a verification pass agrees with the primary one.
def decide(action, ctx):
if not action.reversible and action.blast_radius > 1:
return Gate.APPROVE # irreversible + broad: always ask
if action.confidence < 0.75:
return Gate.APPROVE # unsure: ask
if action.reversible and action.blast_radius == 1:
return Gate.AUTONOMOUS # small + undoable: just do it
return Gate.NOTIFY # act, but make it visibleThe exact thresholds are product-specific and should be tuned against real outcomes, but the shape is durable: combine reversibility, blast radius and calibrated confidence into an explicit gate decision, and log every decision so you can audit and re-tune it.
Design the handoff, not just the stop
Deciding to involve a human is half the work. A handoff that interrupts the wrong person, at the wrong time, with too little context, trains people to rubber-stamp — and a rubber-stamped approval is worse than no approval, because it manufactures false assurance. Good handoffs share a few properties:
- Rich context, low effort. Show the human what the agent wants to do, why, what evidence it used, and what happens if they approve or reject — without making them dig. The decision should take seconds, not minutes.
- The right reviewer. Route to someone with the authority and knowledge to actually judge, not whoever is nearest. A finance action goes to finance.
- A real "no". Rejection must be as easy as approval, and the agent must handle it gracefully — replan, ask for guidance, or stop. If "no" breaks the flow, people stop saying it.
- Feedback that teaches. Every human correction is a labelled example. Capture it, feed it into evaluation, and use it to sharpen where the line sits next time.
Beware the automation traps
Two well-documented human-factors failures show up constantly in agentic products. The first is automation complacency: when a system is right ninety-nine times, the reviewer stops truly checking the hundredth, which is the one that mattered. Counter it by reserving human review for the cases where it genuinely adds value — the low-confidence, high-stakes minority — rather than flooding people with trivial approvals that dull their attention.
The second is the handoff cliff: an agent runs autonomously until it suddenly cannot, then dumps a cold, context-free problem on a human with no time to reconstruct what happened. Avoid it by keeping humans adjacent to the loop for consequential workflows — visible progress, an audit trail, the ability to step in before the cliff rather than only at it.
Start narrow, earn autonomy
The safest path to a trusted agent is to start with more oversight than you think you need and remove it as the evidence accumulates. Launch a new capability at "approve before acting." Watch the approval rate. When a class of action is approved unchanged thousands of times, you have earned the data to promote it to "notify" or full autonomy — with confidence grounded in production, not optimism. Autonomy is not a starting assumption; it is a privilege the system earns one measured step at a time.
The goal was never to remove humans. It was to spend human attention where it changes outcomes and to stop wasting it where it does not. Draw the line by reversibility, blast radius and calibrated confidence; design a handoff worth respecting; and let the agent earn its autonomy. Do that, and human-in-the-loop stops being a safety tax and becomes the reason the system can be trusted to run at all.