Researchers at Google Cloud AI Research and Northwestern University describe Harness², a method that edits an AI agent’s harness while it works on a task, with no answer key to check against, in a paper posted on arXiv on 9 October 2026. One improvement step raised both models they tested on all three benchmarks, by up to 11.5 points on WorkBuddy-Bench, 4.5 on JobBench and 11.8 percentage points on Harvey’s Legal Agent Benchmark, the paper says.
The paper defines the harness as “the prompts, memory, skills, tools, and orchestration through which the model performs tasks”. The models being improved are Google’s Gemini 3.5 Flash and GLM-5; the editor that proposes the changes is Anthropic’s Claude Opus 4.8.
Business Pill · THE MODEL AND THE HARNESS
A one-minute explainer of the model and the harness: the difference between an AI model and the software around it that lets it do work. It teaches the general idea only and says nothing about any person or company in this story.
The key insight: As we read it, Harness² treats an agent’s harness as something to tune per task while the model stays fixed, and the paper’s setup has one lab’s model, Claude Opus 4.8, editing the working environment of another’s, Gemini 3.5 Flash.
The method has two loops. For the current task, a step the authors call Propose–Probe–Compose “proposes harness edits, tests their effects under matched conditions, and composes changes supported by behavioral contrasts”, the paper says.
That step can be run several times in parallel or in sequence. A second loop keeps the adopted harness changes and the lessons from them in a library, which the agent carries into later tasks.
In the experiments the executing models were frozen. The paper says “The default harness editor is Claude Opus 4.8”, running under the open-source OpenCode harness, and the executors ran under OpenCode and OpenAI’s Codex harness.
WorkBuddy-Bench scores for Gemini 3.5 Flash under the OpenCode harness by domain, from Figure 1 of the 9 October 2026 Harness² paper (Google Cloud AI Research, Northwestern): base harness 73.4, 69.9 and 74.0 against Harness² with five parallel candidates at 80.8, 78.3 and 83.4 for code, web and office tasks.
What Was Posted on X
Learn to build custom harnesses, folks!
Here is a highly recommended paper from Google on recursive agent harnessing.
Great insights if you work on evolving your harnesses.
It also offers an important discussion of recursive self-improvement and evolving harnesses for open-world tasks.
They focus on improving your agent harness at test time without a verifier. This allows interesting scaling properties.
Harness^2 proposes several edits to the harness for the current task, tests them against each other, and keeps the combination that works best. Lessons from those edits then update a second harness that the agent carries to future tasks.
Results:
It improved GLM-5 and Gemini 3.5 Flash in every configuration tested across WorkBuddy-Bench, JobBench, and LAB. Running five candidate edits in parallel adds another 2.9 to 3.9 points on WorkBuddy-Bench.
ReasoningBank, a memory baseline, fell up to 6.6 points below the base harness on coding tasks in the same comparison.
DAIR.AI recommended the paper on 15 October 2026 and summarised its results, including the 2.9 to 3.9 points that five parallel candidate edits add on WorkBuddy-Bench and ReasoningBank falling up to 6.6 points below the base harness on coding tasks. The figures in this piece come from the paper itself.
“A decent model with a great harness consistently beats a great model with a bad harness.”
@addyosmani wrote that, and 857K people saw it on X.
Addy spent over a decade on Chrome at Google and now works on Claude Code. When someone who builds these tools says the setup matters more than the model, I listen.
So how do you actually build a great harness? Here’s the roadmap I’d follow if I were starting today:
1. Learn what a harness is Agent = Model + Harness. It’s everything that isn’t the model.
2. Write good instructions The file your agent reads every single session
3. Give it real context Your product, your users, your data
4. Build skills Playbooks it loads only when it needs them
5. Add memory So you never correct it twice
6. Set permissions What it can do alone, and what it never can
7. Connect tools Your files, your terminal, your MCP servers
8. Add checks Grade the agent before a customer does
9. Close the loop So it keeps going until the work passes
10. Put it together Start with your own work, where mistakes are cheap
The roadmap in the image has a real case study for every part, from OpenAI to Ramp.
Every product is getting an agent. This is how you build one that works.
Aakash Gupta posted on 16 October that Addy Osmani wrote “A decent model with a great harness consistently beats a great model with a bad harness”, describing Osmani as a former Chrome engineer at Google who now works on Claude Code, and laid out his own step-by-step roadmap for building an agent harness, starting from “Agent = Model + Harness”. The roadmap is Gupta’s; the view count he cites is his figure.
New Meta paper shows that frontier agents differ widely in how they learn from experience, and the best performer is not the most efficient learner.
Most agent tests measure what a model can do once. They ignore whether it gets better by building tools, notes and skills from past attempts, and at what cost.
They froze model weights and let agents play chess, Go, Hex and NetHack, rewriting their own tools and notes after each round. They then scored held-out games and defined plasticity as held-out gain per dollar spent learning.
In hard chess, Claude Fable 5 rose from 37.5% to 73.3% on held-out games. Across chess, Go and Hex, GPT-5.6 Sol gained 298 points per $1,000 of learning cost, versus 57 for Claude Fable 5.
When you evaluate self-improving agents, track held-out learning curves and cost, and check whether the agent actually uses what it saved.
Rohan Paul summarised a Meta paper (Agent Plasticity, arXiv 2610.08902, posted 6 October) that froze model weights and let agents improve only through the tools, notes and skills they wrote between rounds of chess, Go, Hex and NetHack. In his summary, Claude Fable 5 rose from 37.5% to 73.3% on held-out hard-chess games, while GPT-5.6 Sol gained 298 points per $1,000 of learning cost against 57 for Fable 5. It is the same idea as Harness2: gains that live in the files around a frozen model. Figures are the paper’s as summarised by Paul.
The Results
For Gemini 3.5 Flash under OpenCode on WorkBuddy-Bench, five parallel candidate edits lifted scores over the base harness by 7.4 points on code tasks (73.4 to 80.8), 8.4 on web tasks (69.9 to 78.3) and 9.4 on office tasks (74.0 to 83.4), ahead of the Meta-Harness baseline in each, the paper’s first figure shows.
Raising the number of parallel candidates from one to five took that configuration’s gain from 4.5 to 7.5 points on JobBench and from 5.1 to 8.4 on WorkBuddy-Bench, according to the paper. By contrast, it says, “ReasoningBank falls as much as 6.6 points below the base harness on coding tasks.”
On compute, the authors write that “Meta-Harness uses more compute” than Harness² at five candidates for lower performance, counting both execution and improvement costs.
WorkBuddy-Bench points lost when one component is removed, for Gemini 3.5 Flash under OpenCode with one improvement step (Table 3 of the 9 October 2026 Harness² paper; the full method scores 77.5): library of past edits 3.1, scoped editor roles 2.9, composition 1.6.
What the Ablations Show
Removing the library of past edits lowered WorkBuddy-Bench and JobBench scores by 3.1 and 1.8 points, the paper reports. Removing the scoped editor roles cost 2.9 and 3.7 points, and removing composition cost 1.6 and 1.9.
The authors released code on GitHub under google-research/harness2.
The Structural Read
As we read it, the paper locates part of an agent’s performance outside the model weights: with the executors frozen, editing prompts, memory, skills and tools moved Gemini 3.5 Flash by 7.4 to 9.4 points across WorkBuddy-Bench domains in the authors’ runs.
As we read it, the library is what turns one-off fixes into an asset: removing it cost 3.1 points on WorkBuddy-Bench in the ablation, so lessons from earlier tasks carried measurable value into later ones.
As we read it, the paper adds to agent research we covered today, from Meta’s TreeFlex, which trains the model to steer its own search, to Sakana AI‘s orchestration work, in treating the scaffolding around a model as its own lever.
Harness² paper, Google Cloud AI Research and Northwestern University (9 October 2026)
“Without ground-truth supervision, refinement relies on behavioral contrasts and model judgment”
Three Implications
FOR AGENT BUILDERS The paper reports gains of up to 11.5 points on WorkBuddy-Bench, 4.5 on JobBench and 11.8 percentage points on Harvey’s Legal Agent Benchmark from one improvement step, with frozen models.
FOR INFERENCE BUDGETS The authors write that increasing the recursion budget adds inference cost, and that Meta-Harness used more compute than Harness² at five candidates for lower performance.
FOR READING THE NUMBERS The paper says its feedback signals remain sensitive to execution variability and shared model biases, and do not guarantee that an adopted change transfers to a different task.
The framework argues: “It is an operating system that wraps your AI — a harness that compresses your thinking, your team’s feedback, and the model’s raw capability into something replicable.”
As we read it, Harness² automates part of that compression: an editor model proposes and tests harness changes and files the ones that work in a library for later tasks. The framework was written for professionals building their own harness and does not discuss this paper.
What Is Not Established
The authors say the method costs more to run: “Increasing the recursion budget therefore adds inference cost.” They also write that its feedback signals “remain sensitive to execution variability and shared model biases”.
“They do not guarantee correctness or establish that an adopted change will transfer to a different task,” the paper adds. The results are the authors’ own, reported in a preprint posted on arXiv.
Business Engineer Framework
AI & The Harness Theory
A Business Engineer framework on the harness as the operating layer that wraps a model’s raw capability.
Google researchers’ Harness² uses one model, Claude Opus 4.8, to propose and test changes to another model’s working environment while it runs, and the paper reports gains of up to 11.8 points across three professional benchmarks.
The paper also says the method adds inference cost and does not guarantee that a change will carry over to a different task.
95,000+ executives read Business Engineer for the AI strategy frameworks cited by ChatGPT, Claude, and Perplexity.
A note on sourcing. We read the full text of the Harness² paper, arXiv 2610.14224 (listed 9 October 2026), by researchers at Google Cloud AI Research and Northwestern University; every figure and quotation comes from it. The benchmark results are the authors’ own. We did not contact Google or use other outlets’ reports.
Gennaro Cuofano is a CRO and tech executive who has worked in AI since late 2015, starting with bringing NLP, natural-language generation, voice and chatbot products to the marketing industry. His background is in law and finance: he holds a Master's degree in Law and an International MBA with an emphasis on corporate finance (LUISS Business School and the University of San Diego, 2012), and worked as a financial analyst at a real-estate investment firm in San Diego and as an assistant controller. His work focuses on business model strategy, business engineering and, more broadly, structural analysis: how companies make money, read from their own filings. He created FourWeekMBA and leads research there and at The Business Engineer, his newsletter on AI and business strategy, with over 95,000 subscribers and more than 1,000 published analyses. His writing has also appeared on Entrepreneur (for example, How to Digitize a Family Business in Europe) and HackerNoon. Find him on LinkedIn and Substack. See how we source.
