Human-in-the-loop for incident agents: where the approval gate goes
Put the approval gate where an agent's action is costly or hard to undo. Let an incident agent read freely, ask before it makes a reversible change, and never take an irreversible step alone. Decide each gate by blast radius and reversibility, name who approves, and settle in advance what happens when nobody answers.
Where the gate goes: risk, not features
A human-in-the-loop gate is a point where an agent stops and waits for a person to say yes. The useful question is not “which features need approval?” but “what is the worst thing this action can do, and can it be undone?” Two ideas answer that.
Blast radius is how much an action can affect: one pod or every customer, one internal dashboard or a shared database. Reversibility is how easily the action can be undone: a restart can be repeated, a deleted table cannot. An action with a small blast radius that can be undone in a minute needs little protection. An action with a wide reach that cannot be undone needs the most.
Placing the gate by these two ideas has a practical benefit for the team. The rule is short enough to remember, and it explains itself when someone asks why the agent stopped.
Three classes of action
Most of what an incident agent can do falls into three classes, and each gets a default.
| Class | Examples | Default gate |
|---|---|---|
| Read | Query metrics, search logs, list recent deploys, read a runbook | None, but every call is logged |
| Reversible write | Restart a service, scale up, roll back a release, mute an alert | Ask first, with a time limit on the change |
| Irreversible or wide | Delete data, change a database, rotate a secret, message customers | Ask a named person, or keep it out of the agent |
The line between the second and third class is the one teams get wrong most often. A rollback looks reversible, but if it undoes a migration it is not. Muting a monitor is reversible, but a muted monitor that nobody remembers to unmute is a blind spot. When in doubt, treat the action as the more dangerous class.
Scoring an action: an illustrative example
This example is invented. A team lists the actions its agent could take on a notification service, whose queue has started to back up, and scores each on the two questions.
| Action | Blast radius | Undo | Gate |
|---|---|---|---|
| Read the queue metrics and logs | None | Not needed | None, logged |
| Restart one worker | One worker | Start it again | Ask the on-call engineer |
| Scale up the consumers | The whole service | Scale back down | Ask the on-call engineer |
| Change a feature flag for all users | All users | Flip it back, if recorded | Ask, and show the current value |
| Run a data fix in the database | Shared data | Not reliably | Two people, or a human only |
Two rows show the point. The feature flag is easy to reverse, yet it reaches everyone, so it still asks. The data fix has a narrow purpose but no reliable undo, so it never goes to the agent alone.
Who approves, and what they see
The on-call engineer is the natural approver: they own the incident and hold the context. The owning team of the service should be able to see every action taken on it. For an irreversible action, ask for a second person, so that one tired judgement at night is not the only check.
The request matters as much as the approver. A good one says what will change, on which system, why the agent proposes it, what evidence supports that, and how to undo it. A request that says only “restart the service?” is a click waiting to be given. This is a design problem for the team as much as a technical one: the request is a small interface, and people will use it under stress.
A short checklist for the request:
- The action, in plain words, and the exact system it touches.
- The reason, with the evidence the agent found: the query, the log line, the change.
- The effect, including who or what will notice.
- The undo, and how long it takes.
- The clock: how long the request stays open before it lapses.
Too many requests wear the gate out. When most requests are trivial, people approve by reflex, and the one that matters gets the same reflex. Keep the number of gated actions small, put them where the risk is, and review which ones people always approve.
Timeouts: what happens when nobody answers
Every approval needs a decision about silence. There are only three sensible answers:
- Do nothing. The agent does not act, records that it asked, and reports what it would have done.
- Escalate the request. The request goes to the next person, exactly as an unanswered page would. The escalation policy already knows who that is.
- Expire it. After a set time the request lapses, so an old yes cannot be used against a system that has changed.
What a timeout must not do is turn into approval. An action that runs because nobody said no is not gated. Decide the waiting time, the escalation and the default when the rules are written, not during an incident.
What “autonomous for known patterns” should mean
Some teams want the agent to act alone for problems they already understand. That is reasonable when the pattern is narrow. A safe version has five properties:
- It is written down. A named runbook step, agreed by the team that owns the service.
- It is scoped. One action, on named services, in named conditions.
- It can be undone. Or it is harmless if it runs twice.
- It is limited. A cap on how often it can run, so a loop cannot repeat it forever.
- It can be switched off. By the on-call engineer, in one step, and the switch is logged.
Anything outside that pattern goes back to a person. The permission should also expire and be reviewed, because services change and a rule written for last year’s system may be wrong for this one. What the agent’s tools can reach sets the ceiling on what a gate can protect, which is why the permissions model is worth settling first.
Keeping the gates honest
A gate is a rule, and rules drift. After each incident that involved the agent, look at the approvals: which were given as asked, which were changed, which were rejected, and which were never answered. That record is the evidence for moving a gate. A gate that is always approved unchanged may be too tight. One where people often edit or reject requests shows the agent proposing the wrong things. Change the rules on purpose, in writing, and tell the team.
Frequently asked questions
Does every agent action need a human's approval?
No. If every action needs a click, people approve without reading, and the gate stops protecting anything. Reading metrics, logs and code history can run freely and be logged. Reserve approval for actions that change a system, and be strictest about the ones that cannot be undone.
Who should approve an agent's action during an incident?
By default, the on-call engineer who owns the incident, because they hold the context. Actions on a service should also be visible to that service's owning team. For irreversible actions, ask for a second person. Whoever approves needs to see what will change, why, and how to undo it.
What should happen if nobody answers an approval request?
The safe default is that the agent does nothing and the request escalates, the same way an unanswered page does. An approval that quietly turns into an action after a timeout is not a gate. Decide the timeout, the escalation and the default in advance, not in the middle of an incident.
When can an agent act on its own?
For a narrow, known pattern where the team has already agreed the action in writing: a named runbook step, on a named service, that can be undone, with a limit on how often it runs and a way to switch it off. Everything outside that pattern goes back to a person.
How do we know the gates are in the right place?
Review the approvals. If nearly every request is approved without a change, the gate may be too tight for that action. If people often change or reject requests, the agent is proposing the wrong things. Adjust the rules from that record, and rerun the review after each incident.