Human-in-the-loop for AI agents: when to ask for approval

An AI agent should ask for approval when an action can materially affect a customer, a budget, access rights or production systems. It should not interrupt for every harmless step. Start with a narrow permission boundary, assess impact and reversibility, and require an explicit decision before consequential actions. Approval is one control—not a substitute for security or testing.

What does human-in-the-loop mean?

Human-in-the-loop means placing a person at a meaningful decision point in an automated workflow. The agent proposes an action; an authorised person can approve, change or reject it before execution.

Think of a colleague preparing a purchase order. Reading the catalogue and comparing suppliers does not need your signature. Committing your company to a contract does. The useful question is not “Does this use AI?” It is “What happens if this action is wrong?”

This is one part of taking AI agents from POC to production, alongside evaluation, ownership, costs and adoption.

Four levels of control

The following matrix is an Elia Studio editorial framework, not a regulatory classification. Adapt it to your organisation’s risk appetite and data rules.

Action	Example	Default control	Evidence to keep
Authorised reading	Search documents the user is allowed to see	Execute inside existing access rights; escalate sensitive requests	Identity, source and access event
Preparation	Draft a reply without sending it	Let the agent prepare; show sources and uncertainty	Draft and source references, subject to retention rules
Bounded, reversible change	Add an internal tag within an approved workflow	Consider pre-authorisation with explicit limits, monitoring and tested rollback	Change record, applicable policy and rollback result
Consequential action	Send an external message, pay, delete records or change privileges	Explicit approval before execution; stronger checks for critical actions	Exact approved action, approver and execution outcome

Reading is not automatically harmless. A read can expose confidential material to an external model, cross a department boundary or put malicious instructions into the agent’s context. Access controls, data-location rules and treatment of retrieved text as untrusted input still apply.

OWASP recommends explicit approval for high-impact or irreversible actions, action previews, risk-based autonomy boundaries and audit trails in its AI Agent Security Cheat Sheet. Its Excessive Agency guidance explains why too much functionality, permission or autonomy can turn a model error into a damaging action.

Five questions before allowing an action

1. Who is affected: only the requesting user, another team, a customer or the public?
2. What is the maximum impact: inconvenience, lost data, money, contractual commitment or production outage?
3. Can we undo it completely, and have we tested that reversal?
4. Does it expose personal or confidential data beyond the approved boundary?
5. Is the proposed action within a written policy, or is the agent interpreting an ambiguous instruction?

A draft is reversible. A sent email is not reliably reversible. An internal tag can be low-risk in one system and trigger an automated downstream process in another. Classify the full workflow, not just the tool’s name.

What a useful approval screen shows

“Allow the agent to continue?” is not enough. A reviewer needs the proposed action in terms they can inspect:

• The target: recipient, record, account or environment.
• The exact content or change, including amounts and attachments where relevant.
• The expected consequence and whether reversal is possible.
• The evidence behind the proposal and any unresolved uncertainty.
• A clear choice: approve, edit or reject.

Example: “Send this reply to the supplier?” should show the resolved recipient, complete message and attachments. “Send to the supplier” is ambiguous until the actual contact has been identified.

Approval must be enforced outside the model

An instruction saying “always ask first” is not an execution boundary. The component executing the tool must check the approval independently of the model’s text.

Bind approval to the exact target and parameters. Make it expire, prevent reuse, and invalidate it if the proposed action changes. Check permissions again at execution time: the user may have lost access since approving. For critical actions, consider a second approver or stronger authentication.

OWASP’s agent security guidance explicitly recommends separating proposal from execution, binding approval to the exact action, and using short-lived authorisation with replay protection.

Frameworks can provide the pause-and-resume mechanism. LangGraph interrupts persist workflow state and wait for external input. That mechanism does not decide your policy or replace authorisation checks.

Avoid turning oversight into constant interruption

If users approve every minor step, they eventually stop reading. Put decisions at meaningful boundaries:

• Let a research task complete within its approved data scope.
• Show a complete draft rather than requesting approval sentence by sentence.
• Review a clearly specified batch only if every target and change is visible; otherwise split it.
• Pre-authorise low-risk operations only with explicit limits, revocation, monitoring and tested rollback.
• Escalate exceptions instead of quietly extending permissions.

A predefined policy can reduce prompts. It must not become an unlimited instruction to “do whatever is necessary”.

What we chose for Nemrut’s mailbox pilot

Our Gmail and Infomaniak integration started as an administrator-only, read-only pilot. We designed the control layer to keep credentials outside the model, record access without copying email content into the audit log, and require a confirmation tied to the exact action for writes.

That conservative starting point is appropriate for sensitive mailbox data and an early rollout. It is not a claim that every future low-risk internal change must always require a click. Read the seven questions to ask before connecting a mailbox.

What to test before launch

Test more than the happy path. Reject an action and check that nothing executes. Change the recipient after approval and verify the gate blocks it. Try reusing an approval, revoking access while a request waits, and resuming after a timeout. Test an email or document containing instructions to bypass the reviewer.

Record approval, rejection, expiry and execution outcome without unnecessarily duplicating sensitive content. Define who reviews failures and how to stop the agent. Use our production readiness checklist as a discussion aid, not a certification of safety.

Start with one decision boundary

Choose one workflow and name the action the agent must not execute alone. Specify who approves, what they see, when approval expires and how execution is blocked without it. Then test that boundary before widening scope.

Explore our AI consulting and delivery services, or contact Elia Studio to frame your use case.