How closely should a person supervise an AI agent? The choice is between approving every step and watching the system, and it depends on what a mistake costs.
A short explainer video. The numbers in it are round numbers for illustration.
The cautious way to use an AI agent is to approve every step the agent proposes. A person checks, and only then does the work move on. This is called having a human in the loop.
It is safe, but it has a cost. The agent can work in seconds. The person cannot. So the whole system moves at the speed of the human.
Human on the Loop
The alternative is to step back. The agent does the work on its own. The person watches the overall flow and steps in only when something is flagged. This is called a human on the loop.
A Worked Example
The video uses round numbers. There are 100 tasks, and each takes three minutes to review. In the loop, that is 300 minutes of human time. On the loop, the agent flags 10 that look unusual, and reviewing those is 30 minutes.
The same person can now oversee 10 times the work. But notice what changed. The person is no longer checking the work. They are checking the system. Is it flagging the right things? Is it missing any?
Ten times less review time in this example. The catch is that the person now has to trust what the agent flags. Round numbers for illustration.
When Is It Safe to Step Back?
It depends on two things: how costly is a mistake, and can it be undone. A wrong draft can be fixed. A wrong payment may not.
This is the shift from human in the loop to human on the loop. And it only works if the agent knows when to raise its hand. Deciding what gets flagged is the real design work.
Human in the Loop vs Human on the Loop governs AI Agents. Supervision runs from approving every agent step to watching the flow, chosen by what a mistake costs and whether it can be undone.
Human in the Loop vs Human on the Loop constrains Prompt Injection. The injection defense is fewer powers: the robot may read, but a person approves before it sends, pays or deletes.
Human in the Loop vs Human on the Loop builds on The AI Audit Problem. On the loop, a person reviews only flagged or sampled work, one of the audit lesson’s ways to respond to cheap production.
Earned Autonomy(lesson unlocks soon) builds on Human in the Loop vs Human on the Loop. Earned autonomy moves from tasting every dish to sampling, the shift from human in the loop to human on the loop.
The Business Engineer framework maps where each part of the AI stack sits and who holds the leverage. These pills are short lessons from the same library.
Gennaro Cuofano is a CRO and tech executive who has worked in AI since late 2015, starting with bringing NLP, natural-language generation, voice and chatbot products to the marketing industry. His background is in law and finance: he holds a Master's degree in Law and an International MBA with an emphasis on corporate finance (LUISS Business School and the University of San Diego, 2012), and worked as a financial analyst at a real-estate investment firm in San Diego and as an assistant controller. His work focuses on business model strategy, business engineering and, more broadly, structural analysis: how companies make money, read from their own filings. He created FourWeekMBA and leads research there and at The Business Engineer, his newsletter on AI and business strategy, with over 95,000 subscribers and more than 1,000 published analyses. His writing has also appeared on Entrepreneur (for example, How to Digitize a Family Business in Europe) and HackerNoon. Find him on LinkedIn and Substack. See how we source.
Scroll to Top
Discover more from FourWeekMBA
Subscribe now to keep reading and get access to the full archive.
