OpenAI’s GPT-6-Astra, running in the Codex agent, solved 49 of 50 unpublished research-level theoretical physics problems, a “near-perfect accuracy of 98% at an average cost under USD 3 per problem,” according to a paper by a group of physicists posted to arXiv on 9 October 2026.
Anthropic’s Opus 5.5 in Claude Code solved 46 of the 50. In several cases the paper says it was the experts’ own answers that were wrong, and Astra’s that were right.
Business Pill · BENCHMARKS
A one-minute explainer of benchmarks: why a test score is not your own result. It teaches the general idea only and says nothing about any organisation in this story.
The key insight: As we read it, the checking problem has moved to the human side: the authors say that in several cases the experts’ original solutions, not Astra’s, contained the errors.
The paper, titled “TheoPhysEval: Assessing AI agents at autonomous theoretical physics problem solving,” presents “a suite of 50 unpublished research-level theoretical physics problems with verifiable answers.” Problems were supplied by expert contributors, the paper says, and its coordinators include Suvrat Raju of the International Centre for Theoretical Sciences and Praneeth Netrapalli of Google DeepMind.
According to the abstract, “a typical problem in our collection could be solved by a world expert in about a day or less but would take considerably longer for experts in adjacent specialties.” The agents worked autonomously, without human hints.
The paper tested five agents, each paired with an agent harness: GPT-6-Astra with Codex, Opus 5.5 with Claude Code CLI, Gemini 3.8 Flash with Antigravity, DeepSeek-V4.1-Flash with Codex and GLM 5.3 with Claude Code CLI. Each ran on two tracks: a realistic track with web search and local tools, and an isolated track with web search disabled.
Problems solved out of 50 by each agent with web search (realistic track) and without it (isolated track), per the TheoPhysEval paper (arXiv 2610.16733, submitted 9 October 2026), Figs. 3 and 4; the paper ran the suite once per agent.
On 14 October 2026 @freddier quoted that link and wrote that GPT-6 Astra had insisted the expert results were wrong and that a team had checked and confirmed it. The paper’s own account, quoted in this piece, is that the authors found subtle errors in several of the experts’ original solutions after first marking Astra wrong.
GPT-6 Astra autonomously solves 49 out of 50 unpublished theoretical physics research problems, scoring 98% at under $3 per problem
On 15 October 2026 @haider1 summarised the paper as GPT-6 Astra autonomously solving 49 of 50 unpublished theoretical physics research problems, scoring 98% at under $3 per problem, figures that match the paper’s abstract and results.
The Scores
On the realistic track, Astra solved 49 of 50 problems and Opus 46, according to the paper. Gemini solved 34, and the open-weight models DeepSeek and GLM solved 27 and 21.
On the isolated track, Astra and Opus solved 46 each, Gemini 31, GLM 26 and DeepSeek 25. GLM was the only agent in the paper’s figures that scored higher without web search than with it.
The paper says the full suite was run once per agent “to economize time and costs,” which “might lead to statistical errors in the measurement of the total accuracy.” Tasks that did not finish within 8 hours were counted as producing no output.
When the Answer Key Was Wrong
“It is noteworthy that we initially assessed that Astra had made errors in several additional problems,” the authors write. “However, upon closer examination and consultation with the experts, we found that there were subtle errors in the original solutions. In all these cases, after the experts corrected their solutions, Astra’s solution was found to be correct.”
The paper does not say how many problems that covered. The one problem Astra missed on the realistic track was an inverse problem, which the paper says was not solved by any model.
What It Cost
The paper’s Table III puts Astra’s evaluation cost at $116.28 on the realistic track and $95.20 on the isolated track, inferred from token usage at API rates. Across 50 problems, the realistic figure works out to about $2.33 per problem (our arithmetic), in line with the abstract’s “under USD 3.”
Opus cost $240.43 and $275.46 on the two tracks. The paper says Opus “incurred substantially higher costs than other models.” Gemini cost $95.01 and $90.48, and DeepSeek $37.42 and $36.23.
GLM ran on a coding plan, with an actual cost of USD 180 for both tracks combined, the paper says; at API rates it would have cost $311.71 and $371.46.
What the Authors Read Into It
“On average, agents show only small changes in accuracy when denied web search and access to background research papers, suggesting significant memorization of the literature,” the abstract says.
The authors write that some frontier models may already have surpassed the best human researchers at performing concrete calculations, even in specialized topics. They also discuss “the current lack of agent-led breakthroughs in theoretical physics despite their outstanding problem-solving abilities.”
One reason they give: a good researcher can switch to a related, simpler problem when the first one is too hard, and “current agents are not adept at dynamically switching direction in this manner.”
How the Result Spread on X
The paper was posted and discussed on X on 14 and 15 October. The account @PratikRath15 posted the arXiv link on 14 October, and a post by @haider1 on 15 October summarised it as GPT-6 Astra solving 49 of 50 unpublished problems at under $3 per problem.
An earlier post quoted in that chain, by @freddier, said Astra had insisted expert results were wrong and that a team had checked and confirmed it. We did not find that episode, as the post describes it, in the paper’s text; the paper’s own account is the passage quoted above.
As we read it, the cost line is the businessstory: the paper puts Astra’s realistic-track run at $116.28 for all 50 problems, against $240.43 for Opus.
As we read it, the harness is part of the result: the paper tests each model inside a named agent harness, so its scores describe model-and-harness pairs rather than models alone.
As we read it, the authors separate calculation from research: they write that agents are not adept at switching direction when a problem proves too hard, and they discuss the current lack of agent-led breakthroughs.
The TheoPhysEval authors, in the paper’s abstract (arXiv 2610.16733)
“These results suggest that state-of-the-art agents can accurately perform concrete calculations required across theoretical physics research.”
Three Implications
RESEARCH GROUPS The paper says a typical problem could be solved by a world expert in about a day or less, and puts Astra’s average cost under USD 3 per problem.
MODEL MAKERS The paper reports results for model-and-harness pairs: GPT-6-Astra with Codex, Opus 5.5 with Claude Code CLI and Gemini 3.8 Flash with Antigravity, among others.
BENCHMARK BUILDERS The paper says the authors found subtle errors in several of the experts’ original solutions after first marking Astra wrong.
The framework’s reading rule is that value moves up from the model: “Value is migrating upward from the model toward the harness.”
As we read it, the paper’s design reflects that: every score belongs to a model running inside a named harness, such as Codex, Claude Code or Antigravity. The framework does not discuss this paper.
Evaluation cost for all 50 problems on the realistic track, inferred from token usage at API rates, per the paper’s Table III; GLM 5.3 ran on a coding plan costing $180 for both tracks.
What Is Not Established
The scores come from one run per agent on 50 problems, and the paper itself flags possible statistical error from that design.
The paper is an arXiv preprint. OpenAI did not author it, and the paper does not report a statement from OpenAI or Anthropic about the results.
Business Engineer Framework
The Agentic Harness War
A Business Engineer framework on why value is moving from AI models to the harnesses that run them.
A group of physicists tested five AI agents on 50 unpublished research-level theoretical physics problems, and GPT-6-Astra in Codex solved 49 at an average cost under USD 3 per problem, according to the paper.
Opus 5.5 in Claude Code solved 46, and the paper says that in several cases the experts’ original solutions, not Astra’s, contained the errors.
A note on sourcing. We read the TheoPhysEval paper (arXiv 2610.16733v1, submitted 9 October 2026) in full, including its abstract page. Every score, cost and quotation comes from the paper; the per-problem figure of about $2.33 is our arithmetic on its Table III. The posts on X are described as posts. The AlphaProof, Tao and Sakana context comes from our own earlier reporting. We did not use other outlets’ reports.
Gennaro Cuofano is a CRO and tech executive who has worked in AI since late 2015, starting with bringing NLP, natural-language generation, voice and chatbot products to the marketing industry. His background is in law and finance: he holds a Master's degree in Law and an International MBA with an emphasis on corporate finance (LUISS Business School and the University of San Diego, 2012), and worked as a financial analyst at a real-estate investment firm in San Diego and as an assistant controller. His work focuses on business model strategy, business engineering and, more broadly, structural analysis: how companies make money, read from their own filings. He created FourWeekMBA and leads research there and at The Business Engineer, his newsletter on AI and business strategy, with over 95,000 subscribers and more than 1,000 published analyses. His writing has also appeared on Entrepreneur, HackerNoon and Search Engine People. Find him on LinkedIn and Substack. See how we source.
Scroll to Top
Discover more from FourWeekMBA
Subscribe now to keep reading and get access to the full archive.
