A paper by researchers at Princeton University and Meta FAIR, posted to arXiv in October 2026 and highlighted on X on 29 October 2026, reports that an AI agent which turns each mistake into a small trained adapter lifted its success on the GAIA benchmark from 38.8% to 55.8%.
The paper calls the method Self-Direction. Its abstract says the agent “reflects on its own trajectory in order to extract a lesson and immediately distills it into a lightweight low-rank adapter”, then saves it in a library and loads it again on related tasks.
Short explainer · AI agents that learn from mistakes
A vertical news Short narrated from the headlines and opening lines of this piece and four related pieces.
Business Pill · The Model and the Harness
An evergreen Business Engineer explainer on the model and the harness. It teaches the general idea only and does not discuss this paper.
The key insight: As we read it, the paper moves the lesson an agent learns from its prompt into its weights: each fix becomes a small file that can be stored, routed and loaded again, rather than a note the agent may not reread.
The paper is titled “Don’t Forget What You Just Learned: Self-Directed Repair Turns Agent Mistakes Into Lasting Lessons,” by Yuval Kansal, Karen Ullrich, Polina Kirichenko, Ilia Kulikov, Niraj K. Jha, Mohammad Pezeshki and Mark Ibrahim. It lists Princeton University and Meta FAIR as affiliations.
Its starting point, in the abstract’s words: “in-context reflections are easily forgotten across sessions and compete for limited context, while full-parameter fine-tuning is expensive and can interfere with existing capabilities”.
The agents were built on open-weight Qwen models and tested on three benchmarks: GAIA for open-ended research, Claw-Eval for tool-use workflows and Terminal-Bench 2.1 for terminal tasks.
Mean task success for the paper’s Qwen3.6-27B agent, base versus Self-Direction, on GAIA, Terminal-Bench 2.1 and Claw-Eval, with the gains in percentage points the paper reports. Source: Kansal et al., Princeton University and Meta FAIR, arXiv 2610.31846, October 2026.
What Was Posted on X
Nice new paper on continuous improvement of AI agents from Princeton and Meta
Finds that agents repeat fewer mistakes when each fix is saved as a LoRA adapter and reused on similar tasks.
Agents can fix a mistake within a session, then often repeat it in the next one. Notes in context get forgotten, and full fine-tuning is costly and can disrupt other skills.
With Self-Direction, a failing agent reflects on its run and trains the lesson into a LoRA adapter about 0.02% of the model’s size. A router later loads the closest adapter when the agent fails a similar task.
With Qwen3.6-27B, task success on GAIA rose from 38.8% to 55.8%, beating in-context reflections and a Skills.md file. Across 185 earlier mistakes, 67.3% of repairs still worked by the third encounter, against 42.9% for plain retrying.
If your agents keep making the same mistakes, save each fix as a small adapter they can reload, not only as prompt notes.
Rohan Paul on X, 29 Oct, on the Princeton and Meta paper: with Self-Direction, a failing agent reflects on its run and trains the lesson into a LoRA adapter about 0.02% of the model’s size; with Qwen3.6-27B, task success on GAIA rose from 38.8% to 55.8%.
Self-improving harnesses are one of the most discussed ideas in AI right now.
(bookmark it)
This work proposes a new harness, RuHarness, that beats the strongest harness-evolution baseline by 4.5 points on held-out tasks with Qwen3.5-35B-A3B, and by 5.3 points with GLM-4.7-Flash.
The work targets a gap in current methods. They evolve the harness but keep the criteria that guide each edit fixed, so the criteria stop matching the agent’s weaknesses after a few rounds.
RuHarness evolves the rubric along with the harness. Each round, it finds behavioral criteria in the trajectories, re-scores them, and directs harness edits at the most relevant unresolved weakness. Criteria that stop firing go dormant and return if the weakness reappears.
If your harness evolution stops improving after a few rounds, update the critic too.
DAIR.AI highlights a related paper on X, RuHarness, which it says evolves the rubric that guides each edit along with the agent harness. Per the post, it beats the strongest harness-evolution baseline by 4.5 points on held-out tasks with Qwen3.5-35B-A3B and by 5.3 points with GLM-4.7-Flash.
The Results, by Benchmark
The paper reports that mean success rose from 38.8% to 55.8% on GAIA, from 51.7% to 65.5% on Terminal-Bench 2.1 and from 64.4% to 73.3% on Claw-Eval, gains of 17.0, 13.8 and 8.9 percentage points. By our arithmetic, the GAIA gain is about 44% in relative terms.
On GAIA, the paper compares four ways of repairing mistakes. It gives 49.7% for three attempts (pass@3), 45.5% for keeping reflections in context, 53.3% for accumulating them in a Skills.md file and 55.8% for Self-Direction.
The paper says that on Terminal-Bench 2.1 its 27B-parameter Qwen agent outperforms GLM-5.1, a model it describes as having about 30 times more parameters, and is competitive with closed-weight systems including Claude Opus 4.7 and Gemini 3.1 Pro.
Mistakes That Stay Fixed
The paper tracked 185 earlier mistakes across the three benchmarks. It reports that after one repair, 76.9% of Self-Direction repairs still worked by the second encounter and 67.3% by the third, against 52.4% and 42.9% for the base agent. It also reports that the share of those mistakes solved in all three encounters rose from 9.7% to 18.9%, which the abstract describes as “nearly doubling the reliability of an agent’s corrections”.
By our arithmetic, 67.3% against 42.9% is about 1.6 times as many repairs holding by the third encounter.
The abstract also says that “a handful of adapters generalize to dozens of new tasks without additional training”. The paper reports that a library capped at three adapters, 256 MB in total, raised GAIA success from 36.4% to 52.7%.
What Each Lesson Costs
The paper says each adapter trains extra parameters equal to at most about 0.02% of the base model’s count, takes about 85 MB of memory for Qwen3.6-27B, and takes about 1.5 minutes to optimise on one H200 GPU in its setup, with reflection and example writing adding more time.
By our arithmetic, a library of 100 such lessons would take about 8.5 GB at that size; the paper does not test a library that large.
For comparison, Amazon’s 29 October earnings release said its Strands harness “lowers the cost of agent tasks by 32% through greater token efficiency than other harnesses”, a claim covered in our Bedrock piece; the paper does not discuss Amazon.
The Structural Read
As we read the comparison, the paper puts three places to keep a lesson side by side: in the context window, in a Skills.md file the harness loads, and in an adapter trained on the model; on GAIA it reports the adapter route scored highest of the four repair strategies it tested.
As we read the cost section, the paper prices each lesson in megabytes and GPU minutes, about 85 MB and about 1.5 minutes of adapter training on one H200 in its setup, which is the unit an enterprise would budget for if every customer kept its own library of lessons.
As we read its limitations, the trade-off is portability: the paper says textual lessons transfer across model families and stay readable by people, while in-weight repairs remain model-specific.
Kansal et al., Princeton University and Meta FAIR, arXiv 2610.31846
“Ultimately, this self-directed repair transforms everyday mistakes into lasting lessons.”
Three Implications
FOR TEAMS RUNNING AGENTS The paper says a mistake must first be detected, through self-evaluation, grader feedback or a user report, before a lesson can be trained, so the method depends on catching failures.
FOR PLATFORMS HOSTING AGENTS The paper says training adapters needs GPU resources and storage beyond standard inference, and that textual reflections can be stored until those resources are available.
FOR WHAT TO WATCH The paper reports results on open-weight Qwen models and research benchmarks; it does not report tests with closed models or production deployments.
The framework puts it as “The surface, not the model, governs autonomy. The agentic surface — the harness — is becoming where value pools in the entire AI stack.”
As we read it, the paper tests where a lesson should live, in the harness as a Skills.md file or in the model as an adapter, and reports the adapter scored higher on GAIA. The framework was published in July 2026 and does not discuss this paper.
The results are the authors’ own, on research benchmarks, with open-weight Qwen models; the paper does not report tests with closed models or in production deployments. Its limitations section says the method “begins with a mistake”, that “Encoding that lesson in model weights requires GPU resources and storage capacity beyond standard inference to train LoRA adapters”, and that in-weight repairs “remain model-specific”. This piece rests on the arXiv paper and Rohan Paul’s post on X.
This Week In The AI Supercycle
Execution Fault Lines, Concentrated Demand, and Geopolitical Firewalls
This week’s issue: the AI build is not short of demand for its parts. It is short of cheap money, and its end revenue rests on a narrow base of large, often venture-funded buyers.
Princeton and Meta FAIR researchers report that an agent which trains each fix into a small adapter scored higher on GAIA than one that keeps reflections in its context or in a Skills.md file, and kept more of its repairs than a base agent that simply retries.
The paper puts the cost at about 85 MB per lesson and about 1.5 minutes of adapter training on one H200 GPU, plus time for reflection.
A note on sourcing. The results and quotations come from the arXiv paper 2610.31846 by Princeton University and Meta FAIR researchers, read in full on 29 October 2026; Rohan Paul’s post on X is context. The Amazon passage comes from Amazon’s 29 October 2026 earnings release.
Gennaro Cuofano is a CRO and tech executive who has worked in AI since late 2015, starting with bringing NLP, natural-language generation, voice and chatbot products to the marketing industry. His background is in law and finance: he holds a Master's degree in Law and an International MBA with an emphasis on corporate finance (LUISS Business School and the University of San Diego, 2012), and worked as a financial analyst at a real-estate investment firm in San Diego and as an assistant controller. His work focuses on business model strategy, business engineering and, more broadly, structural analysis: how companies make money, read from their own filings. He created FourWeekMBA and leads research there and at The Business Engineer, his newsletter on AI and business strategy, with over 95,000 subscribers and more than 1,000 published analyses. His writing has also appeared on Entrepreneur (for example, How to Digitize a Family Business in Europe) and HackerNoon. Find him on LinkedIn and Substack. See how we source.
