27B Model Edges Claude Opus 4.8 on PC Tasks for $1.10 Each

A 27-billion-parameter open model fully solved 21.3% of the desktop tasks in the OSWorld V2 benchmark, just above the 20.6% the paper reports for Anthropic’s Claude Opus 4.8, at an estimated $1.10 per task against $72.40. That is according to a paper by researchers at Simular, the computer-use agent company, posted on arXiv on 20 October 2026 and described on Simular’s blog on 21 October.

The model itself was not retrained: it ran in a different agent harness, with tools and interfaces rewritten for it by a second agent.

The paper calls the system Agent S4 and its approach Evolving Agent Interfaces. “We argue that interface design should itself be an agent capability,” the authors write. Simular’s post is direct about the scope: “Agent S4 does not solve tasks itself.” A second agent watches the first one work and rewrites the toolkit it uses.

Business Pill · COST PER TASK

A short Business Engineer explainer on cost per task: what it costs to get a job finished, not the price of one word. It teaches the general idea only and says nothing about the figures in this piece.

What the Paper Found

• Fully solved tasks. On all 108 OSWorld V2 tasks, Qwen3.8-27B with the evolved interfaces fully solved 21.30%, against 20.60% for Claude Opus 4.8 and 13.00% for GPT-5.5 with batched actions, the paper reports. The authors note that “Binary gains are the hardest to obtain on OSWorld V2.”
• Partial progress. On partial score, which credits progress short of a finished task, Claude Opus 4.8 scored 54.80 and GPT-5.5 49.50, against 46.90 for this run and 51.06 for a second evolved run, Table 2 shows.
• Cost. The paper puts the 27B run at about 4.3% of GPT-5.5’s cost per task and 1.5% of Claude Opus 4.8’s. Its costs are estimated from token use at the inference provider Parasail’s rates, assuming an 80% input-cache hit rate, and “exclude repository construction and runtime infrastructure.”
• Same model, fixed interfaces. Run through the Qwen CUA setup with fixed, human-designed interfaces, the same 27B model fully solved 5.56% of tasks, the paper reports. The evolved result is 3.8 times that.
• Scientific software. On ScienceBoard, 169 tasks in six scientific applications such as ChimeraX and Lean, success rose to 58.58% with the evolved interfaces from 41.4% for Qwen CUA and 55.03% with the starting toolkit, solving 99 tasks against 93.

Source: Agashe et al., ‘Evolving Agent Interfaces’, arXiv 2610.28601, 20 October 2026, Table 2. All figures are the paper’s. The Claude Opus 4.8 and GPT-5.5 rows are results the paper reports from the OSWorld V2 reference evaluations, not runs in the same harness. Costs cover task-solving model calls only.

What Was Posted on X

The X account @alex_verem summarised the paper, writing that a 27B open-source model beat GPT-5.5 and Claude Opus 4.8 at finishing computer tasks, at 1.5% of Opus’s cost, and that the $1.10 per task leaves out the cost of building the tools.

How Agent S4 Works

An interface, in the paper’s sense, is how an agent sees a computer and acts on it: clicks and screenshots, or a shell that takes commands and returns text, or a tool that writes a whole block of spreadsheet cells in one call. Simular’s post says that “The task agent’s model and harness stay fixed. Only the repository changes.”

A meta-agent reads the task agent’s runs on synthetic training tasks, which carry no success labels, and writes new or revised interfaces and usage notes into a repository. Candidate repositories compete in a population; the ones that score better on separate validation tasks, which the meta-agent never sees, are developed further. On OSWorld V2 without a scratch machine to test on, evolution added 4 new interfaces and 127 extensions to existing ones, Simular’s post says.

The paper’s case study is an expense claim. With fixed interfaces, the agent “opens receipt emails one at a time, handles five browser download warnings”, and finding one card charge took “eight scrolls and four clicks”. With the evolved interfaces, it read the mailbox and the bank records as data through an interface that sends requests from the already logged-in browser tab, the paper says.

Where the Gain Came From

The paper’s Table 4 separates the steps for the same Qwen3.8-27B model. Moving from the Qwen CUA setup to the Prime Agent harness, which runs code in a persistent Python kernel, with no repository at all, lifted full solves from 5.56% to 15.74%. The 15-interface seed toolkit, about 550 lines of code, lifted that to 19.44%, and evolution to 21.30%.

Source: Agashe et al., arXiv 2610.28601, Tables 2 and 4. The step-by-step differences below are our arithmetic from those tables.

By our arithmetic from those tables, of the 15.74-point rise in fully solved tasks, 10.18 points came with the harness change, 3.70 with the seed toolkit and 1.86 with evolution. The authors’ own summary of that table: “The seed therefore provides a useful starting point, and evolution improves it on binary success and partial score.” The seed also cut the estimated cost per task from $1.36 to $1.11, the paper reports.

Connecting the Coverage

Our harness-scaling piece reported on 23 October that training an agent across 18 harness configurations, instead of one, raised a small open model’s average score from 29.81 to 39.04, in a paper from Renmin University, IQuest Research and Beihang University. Our token-spend piece reported the same day that Peter Steinberger, the creator of OpenClaw, put his token spend last month at around $10 million. What’s new: in this paper, the Agent S4 evolution step changes neither the model nor the harness, only the repository of interfaces the agent works through.

The Structural Read

As we read it, the widest gap is on cost: the paper estimates $1.10 per task for the 27B run, $25.50 for GPT-5.5 and $72.40 for Claude Opus 4.8, and says its own estimate excludes building the interfaces and runtime infrastructure.

As we read it, most of the jump came before any evolution: by our arithmetic from Tables 2 and 4, the harness change added 10.18 points of fully solved tasks and the 15-interface seed toolkit 3.70, while evolution added 1.86.

As we read it, the interfaces are the part a company can own: Simular’s post says the task agent’s model and harness stay fixed and only the repository changes, and the code is open source in the Agent S repository.

Three Implications

FOR COMPANIES DEPLOYING AGENTS The paper prices a full OSWorld V2 task at an estimated $1.10 for the 27B run, task-solving calls only; as we read it, interface work is a cost lever alongside the choice of model.

FOR MODEL BUYERS On fully solved tasks the 27B run edged the reported Claude Opus 4.8 result, 21.30% to 20.60%, but trailed it on partial score, 46.90 to 54.80, Table 2 shows.

FOR EVALUATORS The closed-model rows are results the paper reports from the OSWorld V2 reference evaluations with batched actions, not runs in Simular’s harness.

The Business Engineer Lens

This story maps onto the Business Engineer framework The Agentic Harness War.

The framework argues that “The surface, not the model, governs autonomy.” It contrasts two bets: “OpenAI is betting on one harness that absorbs everything” and “Anthropic is betting on many harnesses, embedded where work already lives.”

As we read it, this paper puts numbers on the surface side of that argument: the same 27B model went from 5.56% to 21.30% of tasks fully solved as its harness and interfaces changed. The framework was published in July 2026 and does not discuss this paper.

What Is Not Established

The Claude Opus 4.8 and GPT-5.5 figures are results the paper reports from the OSWorld V2 reference evaluations, under batched-action settings, not runs in Simular’s harness. The cost estimates cover task-solving model calls only; the version we read gives no dollar figure for the 30 construction steps of evolution. The authors write: “We do not run the denoising step for OSWorld-v2 due to much longer time and cost of evolution and repeated evaluations.”

The work was done at Simular, which sells computer-use agents. We read the 20 October version of the paper and Simular’s 21 October post, did not run the open-source code and did not contact the authors. Nothing here is investment advice.

The Bottom Line

In the paper’s tests, a 27B open model fully solved slightly more OSWorld V2 tasks than the reported result for Claude Opus 4.8, at an estimated $1.10 per task, after its interfaces were rewritten by another agent. Most of the rise from the fixed-interface baseline came with the harness and the starting toolkit; evolution added the last 1.86 points, by our arithmetic.