Responsible use · 7 min read
Field note 012Responsible use
Before you approve an AI action
In one approval game, players missed roughly one in three threats when approving AI agent commands. Learn why human-in-the-loop fails in practice. A practical check can help reviewers pause and examine the action. A quick checklist is not proof of safety.
01The comfort of the Allow button
Every serious AI tool now puts a human in the loop. The agent wants to send an email, run a command, or update a record — and a screen appears: Allow?
That screen is where all the safety promises live. The vendor points at it. Your IT policy points at it. You point at it, when someone asks whether the AI could do something wrong: "It can't. I approve everything."
The uncomfortable question is whether that approval is real, or whether it is theatre. Recent evidence says: at normal working speed, it is theatre more often than anyone would like.
02What the study actually found
Scalex, a software engineer's blog, published a browser game in which people approve or reject commands proposed by an AI coding agent under time pressure, then reported the results of roughly 40,000 plays (results, discussion).
The headline result: the average player missed about one in three threats. The dangerous action was on screen. The reviewer had the power to stop it. They clicked approve anyway.
The honest caveats first. It is one short browser game, played under artificial time pressure, and it has not been independently audited. Treat the number as a prompt, not a statistic.
But the direction should surprise nobody who has ever clicked through a cookie banner. Showing a human a confirmation screen is not the same as the human confirming anything.
03Why smart people approve bad actions
Approval fatigue is not a character flaw. It is what happens when screens are written for the tool's convenience instead of yours.
Three forces do the damage:
- Repetition. The tenth "Allow?" of the morning gets the same click as the first — and the tenth is the one that sends the wrong draft to the wrong person.
- Framing. Most approval prompts show the technical how (a wall of parameters) and hide the consequential what ("this sends an external email to 40 recipients").
- Default momentum. Approve is the button that makes work continue. Reject is the button that makes work stop. At 4:45pm, that is not a fair fight.
None of this is fixed by telling people to be more careful. It is fixed by changing what a person must be able to say before they click.
04A worked example: the tenth approval of the morning
Picture an operations executive using an AI assistant that drafts and sends supplier emails.
Nine times this morning, the screen said the same thing: Agent wants to send email. Allow? Nine times, the email was routine. Nine approvals, each faster than the last.
The tenth request looks identical — but this draft quotes the confidential rate card, and the recipient field says the wrong supplier.
The information needed to catch it was technically on screen, two taps deep, in a preview nobody opens by the tenth time.
The reviewer did not fail. The screen did. It trained them, nine times in a row, that reading was unnecessary.
05The 10-Second Approval Test
Here is the practice. Next time an AI tool asks you to approve an action, answer four questions before you click. Out loud if you like. Ten seconds.
- Action
Can I say what it is about to do in plain words, without reading the prompt again?
- Blast radius
If this is wrong, who sees it and what does it touch? Me only? A client? Money?
- Scope
Is it asking for this one action, or for standing permission that outlives the task?
- Undo
Do I know how to reverse it — and is reversal actually possible once it is out?
If you cannot answer all four in ten seconds, do not approve. Reject, and make the tool restate the action until you can.
The rejection is not a failure state. It is the system working: an action you could not summarise is an action you were about to rubber-stamp.
The 10-Second Approval Test
- 01Action
Can I say what it is about to do in plain words, without reading the prompt again?
- 02Blast radius
If this is wrong, who sees it and what does it touch? Me only? A client? Money?
- 03Scope
Is it asking for this one action, or for standing permission that outlives the task?
- 04Undo
Do I know how to reverse it — and is reversal actually possible once it is out?
06The screen design that produces the misses
Once you run the test a few times, you start noticing which tools make it easy and which make it impossible.
The pattern to watch for: any approval screen that shows you the HOW — a wall of technical detail, a raw command, a JSON blob — but not the WHAT, the plain-language consequence. That design gap is where the one-in-three lives.
A good approval screen answers the four questions before you ask them: plain action description, named recipients or systems affected, one-time versus standing scope, and the undo path. If a tool you rely on cannot show you that, the ten-second test is your compensating control — and worth raising with the vendor.
07Building the habit in a team
One person running the test protects one inbox. A team habit protects the workflow. Three ways to make it stick:
- Weekly autopilot review. Once a week, look at the approvals you granted on autopilot. If you cannot remember approving something, that approval was theatre — discuss it, do not hide it.
- Reject without penalty. Make it explicit that rejecting an AI action to demand a clearer restatement is good work, not slow work.
- Escalate the blanks. When nobody on the team can answer the Undo question for a tool, that is a finding for whoever owns the tool relationship — before an incident, not after.
A human in the loop only counts if the human is actually in the loop. Approval is a skill, not a click — and the ten seconds it costs is the cheapest risk control your team will ever buy.
08Frequently asked questions
It means an AI system is designed so that a human reviews or approves certain actions before they take effect — sending a message, executing a trade instruction, changing a record. It is the most common safety control in workplace AI. Its weakness is that it assumes the review actually happens, rather than becoming a reflexive click.
Only when the oversight is designed to be performable. Evidence like the Scalex approval game — players missing roughly one in three threats across 40,000 plays — suggests that a bare confirmation dialog does very little. Oversight works when the screen states the action plainly, shows consequences, limits scope, and offers an undo — or when the reviewer imposes that standard themselves.
The decay in attention that comes from approving many similar requests in a row. By the tenth identical-looking prompt, most people click from pattern memory, not from reading. It is a design problem: screens that front-load technical detail and hide consequences train users to stop reading.
Action (can I state what it will do in plain words?), Blast radius (who or what is affected if it is wrong?), Scope (one action, or standing permission?), and Undo (do I know the reversal path, and does one exist?). All four answerable in ten seconds, or you reject and demand a clearer restatement.
Yes, and it should be costless. A rejection that forces the tool to restate an action more clearly is the control working as intended. Teams that treat rejection as slowing things down are training their people to rubber-stamp — which converts every future incident into "but a human approved it."
Treat it as directional. It comes from a single browser game, published on a software engineer's blog, that has not been independently audited. The reason to act on it anyway is that its direction matches everyday behaviour everyone recognises — banner blindness, cookie-consent clicking, terms-of-service scrolling. The test costs ten seconds; being wrong about the exact percentage costs nothing.
