Business Pill · When an agent uses the screen
Computer use is a model operating software through the screen, the way a person does. It reaches old systems that have no connector, and one wrong click is a real click.
A short explainer video, under a minute. The manager, the robot and the screen are an illustration.
At a glance
What Computer Use Is
Then the robot is taught to use the screen itself.
What It Opens Up
Computer use opens every old system to automation. A system with no connector still has a screen, and a screen is enough.
Why It Matters
The video turns this into one question: which screens would we let an agent click on?
The Short Answer
Picture a company whose oldest system has no connector. It is built for people, so the robot cannot reach it.
A person copies the numbers across, by hand, every day.
What Computer Use Is
Then the robot is taught to use the screen itself. The video draws it as a loop: look at the screen, click or type, check what changed.
That ability is called computer use: a model that operates software the way a person does, through the screen.
What It Opens Up
Computer use opens every old system to automation. A system with no connector still has a screen, and a screen is enough.
The Idea in Parts
The lesson in its own parts
What Computer Use Is
Then the robot is taught to use the screen itself.
What It Opens Up
Computer use opens every old system to automation. A system with no connector still has a screen, and a screen is enough.
What It Costs
It is slower than a real connector. And one wrong click is a real click.
What It Costs
It is slower than a real connector. And one wrong click is a real click.
The video shows the robot saying โOops. I pressed delete all.โ The agent clicks on the real system, not on a copy of it.
Why It Matters
The video turns this into one question: which screens would we let an agent click on?
If a person can click it, an agent can too. So decide where it may click.
The key insight: If a person can click it, an agent can too. So decide where it may click.
The honest limit
It is slower than a real connector. And one wrong click is a real click.
The video shows the robot saying โOops. I pressed delete all.โ The agent clicks on the real system, not on a copy of it.
Where This Shows Up in AI Business Today
From our news coverage
ThinkingBox: Opus 5.5 and Opus 5 Tie at 241 of 507 Tasks
Computer use is easiest to judge on the result it leaves behind. A joint Microsoft and Hugging Face post on ThinkingBox, which our report covers, says the benchmark grades AI agents on terminal backend state and side effects across 507 stateful business workflows, each run 20 times.
From our news coverage
The post says its opening example is a retail support case: an agent made nine well-formed tool calls, read the refund policy correctly, and still closed a ticket as solved, when with the carrier exception still open the required end state was on hold. Our report verified none of the post’s figures independently.
The Business Engineer Lens
Where this idea sits in the Business Engineer library:
The Question to Ask
- Which screens would we let an agent click on?
How This Concept Connects
Layer 4 of 8 · Agents & Workflows, on the Business Engineer Concept Map
- Computer Use builds on Multimodal. Computer use loops on looking at the screen, clicking and checking what changed, which needs a model that reads images.
- Computer Use contrasts with MCP. Computer use reaches systems with no connector through the screen, slower than a real connector such as MCP.
- Computer Use builds on AI Agents. Computer use is an agent loop on the screen: look, click or type, then check what changed.
- Least Privilege (lesson unlocks soon) governs Computer Use. If a person can click it, an agent can too, so access must be narrowed to the screens the agent may touch.
Agentic AI Stack
Screen-operating agents in the agentic stack.Business Engineer framework
The Product Overhang Doctrine
Opening legacy systems is latent capability.
Get the full Concept Map as a free poster.
Related Business Pills
All Business Pills
- The Memory Wall: Why a Faster AI Chip Is Not Faster AI
- Tokens per Watt: What an AI Data Centre Actually Produces
- Why AI Can’t Be Both Instant and Cheap: Latency vs Throughput
- The Model and the Harness: Why Same-Model Products Differ
- Context, Not Capability: Why a Smart AI Model Gives Poor Answers
- Discardable Software: When Code Is Cheap Enough to Throw Away
- Human in the Loop vs Human on the Loop: Supervising AI Agents
- The AI Audit Problem: When Making Work Is Cheaper Than Checking It
- AI Evaluation as Acceptance Test: How to Know It’s Good Enough
- Extensibility Is Control: Who Holds the Power in an AI Product
- RLHF Explained: How an AI Model Learns What People Prefer
- Pretraining Explained: How an AI Model Learns Before Anyone Teaches It
- Fine-Tuning Explained: How to Adapt a General AI Model to One Job
- Tokens Explained: The Unit AI Reads, Writes and Bills In
- Embeddings Explained: How a Machine Compares Meaning
- RAG Explained: How a Model Answers From Your Documents
- AI Hallucination Explained: Why Models State False Things
- Distillation Explained: How a Small Model Learns From a Large One
- Reasoning Models Explained: What Changes When AI Thinks First
- Tool Use Explained: How an AI Model Goes From Text to Action
- Capex and Depreciation Explained: Why Chip Lifetime Drives AI Profits
- Run-Rate Revenue Explained: What an AI Company’s Number Means
- Backlog Explained: Revenue That Is Signed but Not Yet Earned
- Switching Costs Explained: Why It Is Hard to Leave an AI Supplier
- Temperature Explained: The Dial That Sets How Predictable an AI Model Is
- Guardrails Explained: Rules Enforced Around an AI Model, Not Inside It
- Benchmarks Explained: Why a Test Score Is Not Your Own Result
- Agent Memory Explained: How an AI Assistant Remembers You Between Chats
- MCP Explained: The Shared Plug That Connects AI Models to Tools
- Quantization Explained: How a Large AI Model Is Shrunk to Fit
- Parameters Explained: Where an AI Model’s Knowledge Lives
- Scaling Laws Explained: Why AI Labs Keep Building Bigger
- Synthetic Data Explained: When a Model Writes Its Own Training Examples
- Transformers and Attention Explained: How AI Reads Every Word at Once






