What does human-in-the-loop mean for AI agents in 2026?
Human-in-the-loop means a person decides named actions before they run, under a policy written in advance. Reads never wait, and an approved call runs once.
Human-in-the-loop for AI agents means a person decides specific actions before they run, under rules written in advance. The usable version gates only the actions the policy names, never gates reads, runs an approved call exactly once with the arguments the person saw, and reaches that person where they already work: Slack, email or a dashboard queue.
manage__approval_wait, not sending it again.
stripe__stripe_api_write · asked by Claude Codestripe__stripe_api_write · asked by Claude CodeWhy do approval prompts turn into theater?
When every action asks, nobody reads the question. A client that prompts before each file write, each shell command and each API call trains its operator to press approve. The prompt still appears. The decision no longer happens. The record shows a human approved the refund, the deploy and the deletion, and it is true in the narrowest sense.
The second failure is placement. A prompt that lands in a terminal session reaches whoever is sitting there, and nobody when the agent runs headless. A question nobody can answer becomes a denial or a retry, and the agent routes around it.
The third failure is scope. Guards built into one client cover that client. Claude Code's auto mode and Codex's "Approve for me" are configurable guards inside one client, and you can forward their prompts to Slack with hooks you build. A support bot built on an SDK and a nightly job you operate have no such guard unless you build one for each.
What does a usable loop look like?
Four properties separate a loop people keep on from one they turn off.
- The policy is written in advance, in sentences, by the person who owns the risk.
- Reads are never gated. Only writes are judged, and only the actions the policy names pause.
- An approved call runs exactly once, with the arguments the person approved.
- The card reaches the person where they are, and whichever channel decides first wins.
CoreSpeed's Smart Approval, in public beta since late September 2026, is built on those four. It is a gate between the tool call and execution, in front of the tools, so it covers every agent and client on the team with one rulebook. It sees only calls made through CoreSpeed. It is off until an org turns it on at Dashboard → Approvals; a new org has no policy and no gate.
What are the three verdicts?
Every write an agent makes through CoreSpeed is judged against the policy and gets one of three answers.
| Verdict | What happens | What the agent sees |
|---|---|---|
| Allow | The call runs and is logged | The tool's normal result |
| Ask | The call is frozen; a card goes to a person | A receipt with an approval id |
| Deny | The call is refused | An error, with the reason when a human denied |
The policy is plain English. The example from the docs reads: "Sam handles support. Replies to customers who have written to us first are fine. Ask me before anything goes to a public channel or an external company, and before any refund over $100. Never post on our behalf about pricing, funding, or anything that reads like a product promise."
Calls the policy never mentions fall to a default behavior. Relaxed runs them. Balanced, the default, runs them unless destructive, then asks. Strict asks for every write. Lockdown refuses every write. Written rules beat the default. Two guards sit above every verdict: a low-confidence judgement becomes Ask, and an Allow that looks like prompt injection becomes Ask. A failed evaluation, from a timeout or an outage, goes to a separate Ask or Deny setting, default Ask. No failure path lets a write run.
The judge sees the policy, the tool name and description, its annotations, who is calling from which client, how often this caller used the same tool recently, and the argument values, which an admin can turn off. It never sees the agent's conversation. The judge is Jev, TypeSafe's System One model: it answers typed questions with calibrated probabilities in one pass, a few seconds at most, with thresholds that are deterministic and auditable.
What happens to the agent while it waits?
The held call returns at once with an error-shaped receipt:
Approval required, not run. Call manage__approval_wait with this id.The agent calls manage__approval_wait with that id. The call blocks until the decision and returns the original tool's result, or a denial or expiry error. The agent must not re-send the original call: a fresh tools/call is a new request and can raise a second card. The card expires about thirty seconds after the agent stops waiting, and a card nobody decides expires after an hour. Clients that send a progress token, as Claude Code does, get a progress notification every ten seconds.
Exactly once is a property of the design. The call is frozen when held, the arguments the person approves are the arguments that run, and two simultaneous approvals hand it over once.
Who can decide?
The member who made the call, or the owner of the agent or API key that made it. An agent cannot approve its own action, and an agent's own Slack tools cannot post a card or write into the approvals channel. Admins can see every card, and seeing is not deciding.
Cards reach three places. The dashboard queue is the only place the full call is shown. Email carries the tool and who asked, with approve and deny links and no arguments. Slack carries a card in a DM or one org channel, posted by the CoreSpeed bot in the workspace an admin installs once. Every call lands in the activity trail with its outcome: ok, error, held or denied.
Read the launch post for the fuller picture and the approvals docs for the reference.
FAQ
Does every action go through approval?
No. Reads are never gated. Writes are judged, the actions your policy names pause, and on Balanced an ordinary unmentioned write runs.
Can a member loosen the org policy for their own calls?
No. An admin writes the org policy, and a member can add text that only makes things stricter for their own calls.
Can I test a policy without running anything?
Yes. Test policy on the policy page runs a hypothetical call through the same evaluation without executing it. Every save is a new policy version with history.
What if the agent runs headless and nobody is online?
The card waits up to an hour in the dashboard queue, in email and in Slack. If nobody decides, it expires and the agent gets an expiry error.