Three researchers writing on Anthropic’s Science Blog say AI agents that build scientific models have so far been tested mostly on problems whose answers were already known. “So far, these systems have mostly been tested on rediscovering models that scientists already know. Evidence that they work on problems with no known answer is sparse”, they write in a guest post published on 6 November 2026.
The authors are Jakob Macke, Daniel Gedon and Stefan Wahl, whom Anthropic describes as AI for Science grantees and researchers at the ELLIS Institute Tübingen and the University of Tübingen. By our count, the post cites six AI model-building systems from 2024 to 2026 and presents no new benchmark scores of its own.
Business Pill · AI Agents Explained
A short explainer on AI agents. It teaches the general idea only and does not discuss this post.
The key insight: As we read the post, its authors separate fitting data from explaining it: they write that AI agents can now build and fit candidate models, but that testing so far has mostly been on known answers and that interpretation is the hardest step to hand over.
The post splits scientific modelling into three steps: building a model, constraining its parameters to fit data, and interpreting what it says about the real system. The authors write that “Machines have arguably made the most progress in constraining models”.
For constraining, they describe simulation-based inference, which runs a model many times with random parameters and trains a neural network on the results. They write that for colliding neutron stars, the method “can work out roughly where in the sky the collision happened, fast enough to point telescopes at it while the signal is still detectable”.
For building, they describe LLM-based agents that write code for candidate models, run them, fit them to data and revise them. “Several such systems have appeared in the past two years”, the authors write, and the post’s first footnote lists six examples.
The 12 works cited in the three footnotes of the guest post ‘Teaching AI to build scientific models’ on Anthropic’s Science Blog, 6 November 2026, grouped by footnote: six LLM-based model-building systems (2024-2026), three works on treating models as a distribution, and three fruit-fly wiring-diagram papers. Count and grouping are ours.
What Was Posted on X
New on our Science Blog: How AI is beginning to change scientific modeling.
Scientific models help researchers explain and predict real-world systemsโlike how a single nerve cell fires, or what happens when neutron stars collide. Recent advances have transformed how scientists fit complex models to data. But scientists still typically write the models manually, and usually need to weigh up several competing versions, each of which might explain the data in a different way.
In a guest post, Jakob Macke, Daniel Gedon, and Stefan Wahl describe how AI is starting to help with these parts of the process. By writing, running, fitting, and revising scientific models as code, AI can explore many possible model structures, and find out which explanations the data support.
Anthropic posted the guest post on X at 19:49 UTC on 6 November, saying that by writing, running, fitting and revising scientific models as code, AI can explore many possible model structures and find out which explanations the data support. The caveats this piece leads with are in the post itself, not in Anthropic’s X summary.
The Caveat on Unknown Problems
The authors open their caveat with “We shouldn’t get too far ahead of ourselves”, then note that the testing so far has mostly been rediscovery of known models. They add that “newer LLMs perform better at research tasks than older ones, and we expect these systems to keep improving”.
They also say a fitted model is not the same as understanding. “Run an agent twice on the same data, and it might return two different, and equally good, models”, they write, and “An agent that returns only one of them is hiding the rest, just as a single best-fit set of numbers does”.
Their proposed fix is to ask an agent for the whole range of models that could plausibly explain the data, weighted by how well each is supported, rather than a single best fit.
Interpretation Stays Hard
“Of the three steps, interpretation is the hardest to hand over to AI”, the authors write. They say we can measure how well a model fits data, but not whether it leads to a better understanding of the system.
The post’s worked example is the 1951 Hodgkin–Huxley model of a squid nerve cell, which Huxley solved by hand in over three weeks. The authors write that “An AI system could now plausibly do much of the same work in an afternoon”, but argue its value lay in knowing which simpler models had failed.
Our Numbers
By our count, the post’s three footnotes cite 12 works: six LLM-based model-building systems published from 2024 to 2026, three works on treating models as a distribution, and three papers on the fruit-fly brain’s wiring diagram. Half of the cited works, by our arithmetic, are AI model-building systems. The post itself reports no new benchmark results.
As we read the caveat, it turns on the test set: the authors write that the systems have mostly been tested on rediscovering models scientists already know, and that evidence on problems with no known answer is sparse.
As we read the two-runs point, the authors treat a single returned model as incomplete: they write that two runs on the same data can return two different, equally good models, and propose asking for the whole range of plausible models instead.
As we read the Hodgkin–Huxley example, the authors place the value of a model in what was ruled out: they write that the fourth-power interpretation carried weight only because simpler models had failed.
Jakob Macke, Daniel Gedon and Stefan Wahl, guest post, Anthropic Science Blog, 6 November 2026
“A scientific model may provide an answer, but it is the process of constructing it that provides the insight.”
Three Implications
FOR RESEARCH TEAMS USING AI AGENTS The authors propose asking an agent for the range of models that fit the data, weighted by support, rather than a single best fit.
FOR READING CLAIMS ABOUT AI SCIENCE The authors write that evidence on problems with no known answer is sparse, so a result on a known problem shows rediscovery, in their framing.
FOR COMPARING THIS WEEK’S AGENT STUDIES This post is an essay by outside researchers and tests no model, unlike the measured studies in our scorecard.
The analysis says: “Universal harnesses are likely to dominate highly digital, modular, and verifiable work.”
As we read it, the guest post draws a similar line inside science: the authors write that fit to data can be measured, but whether a model leads to better understanding cannot, which is why they call interpretation the hardest step to hand to AI. The analysis does not discuss this post.
What Is Not Established
The post does not say which tests the six cited systems were evaluated on, or how many of those tests involved known answers; the word “mostly” is the authors’. It is a guest essay, not a peer-reviewed study.
This Week In The AI Supercycle
Anthropic’s Reality Check, The Capital Squeeze, & The New World Order
This week’s issue: for two years the AI build-out was priced as a growthstory. This week it started being priced as a credit story.
Writing on Anthropic’s Science Blog, three Tübingen researchers say AI agents that build scientific models have mostly been tested on known answers, that two runs can return two equally good models, and that interpretation is the hardest step to hand to AI.
A note on sourcing. This piece is based on the guest post by Jakob Macke, Daniel Gedon and Stefan Wahl on Anthropic’s Science Blog, read in full on 6 November 2026. All quotations are from that post. The count of cited works and its grouping are ours. Links to our earlier coverage give context only.
Gennaro Cuofano is a CRO and tech executive who has worked in AI since late 2015, starting with bringing NLP, natural-language generation, voice and chatbot products to the marketing industry. His background is in law and finance: he holds a Master's degree in Law and an International MBA with an emphasis on corporate finance (LUISS Business School and the University of San Diego, 2012), and worked as a financial analyst at a real-estate investment firm in San Diego and as an assistant controller. His work focuses on business model strategy, business engineering and, more broadly, structural analysis: how companies make money, read from their own filings. He created FourWeekMBA and leads research there and at The Business Engineer, his newsletter on AI and business strategy, with over 95,000 subscribers and more than 1,000 published analyses. His writing has also appeared on Entrepreneur (for example, How to Digitize a Family Business in Europe) and HackerNoon. Find him on LinkedIn and Substack. See how we source.
