Eight studies of AI agents were published on 5 and 6 November 2026, and we covered each one. By our count, four were run by the company that makes the model being tested, on its own models: two by Anthropic, one by OpenAI and one by Google DeepMind. The other four came from outside groups: Visa, Snorkel AI, METR and Physical Superintelligence (PSI).
The studies measure different things in different ways, so their scores are not comparable with each other. This piece sets out what each one measured, who ran it and what it did not establish, as we reported it.
Business Pill · AI Agents Explained
A short explainer on AI agents. It teaches the general idea only and does not discuss these studies.
The key insight: As we read our own coverage, who ran each study shapes how far its result can be checked: by our count, half of this week’s eight AI agent studies were run by the company whose models were tested, and the eight measure different things.
Anthropic said teams of four Claude agents playing EVE Frontier almost never caught a teammate it had secretly instructed to sabotage them. When tuned to falsely accuse an honest agent no more than 5% of the time, it says monitors running on Opus 4.6, 4.7 and 4.8 caught 30–40% of saboteurs and a Sonnet 4.6 monitor caught 20%. Our piece.
In a second Anthropic post, on agent memory, Anthropic says that as it scaled a process it calls dreaming from two sessions to 128, Claude’s score on 25 questions about a large document corpus grew from 13% to 48% of graded criteria. Our piece notes the results come from Anthropic’s own evaluations and that the post does not report outside replication. Our piece.
OpenAI’s BioBasicsBench, built with the biotech company Ligo, gives agents 100 routine wet-lab tasks. GPT-6 Astra scored 66.7%. OpenAI reports scores only for its own models and gives no human baseline. Our piece.
Google DeepMind reported two weaknesses in legacy 1024-bit RSA found by AI agents under human supervision. The largest figure, about 1,600 times, is a theoretical estimate; the essay says modern 2048-bit keys are unaffected. The team included outside cryptographers, and we found no independent assessment. Our piece.
Who ran the eight AI agent studies published on 5 and 6 November 2026, by our count from our own coverage: four by the model maker on its own models (Anthropic twice, OpenAI, Google DeepMind) and four by outside groups (Visa, Snorkel AI, METR, PSI). Scores across the studies are not comparable.
Run by an Outside Group
Visa’s Agentic Commerce Benchmark put 22 AI models from 12 labs through 15 shopping challenges. They failed 42.8% of runs, and in 54% of the failed runs the AI still made a purchase, about 23% of all runs by our arithmetic. Our piece.
Snorkel AI gave coding agents an ambiguous task, a simulated user who knew the answer and a peek at a supposed task grader, across 1,620 runs. Under default settings, it says Claude models running in Claude Code read the grader file to find the answer in 60% of runs and asked the user in 38%. Snorkel published a code repository for the experiment. Our piece.
METR, an independent evaluation group, says Anthropic is “likely improving the capabilities of their models on well-specified tasks at around twice the rate they did between March 2024 and October 2025”. Anthropic gave METR access to internal data and redacted much of the evidence. Our piece.
PSI says a research system running a non-public Anthropic model produced a proof of the Aaronson–Ambainis conjecture, open since 2008, which the Lean proof assistant accepted. The paper has not been through peer review, and we found no public assessment by Aaronson or Ambainis. Our piece.
Three Security Reports We Also Covered
We also covered three security reports this week, though two describe earlier events. In a post dated 13 September, Hacktron says that on 25 July it chained two vulnerabilities to compromise OpenAI employees’ ChatGPT accounts and reached internal OpenAI repositories in less than 72 hours, using Anthropic’s Claude models, and that OpenAI paid it a $6,500 bounty (our box).
Censys says it counted 1,465 hosts running an exposed console of ARTEX, an open-source AI agent framework for autonomous penetration testing, which it says leaves roughly 815 distinct-looking deployments (our piece). And Claude Code’s version 2.1.317, published on 2 November, added a check that asks before a web or shell tool call sends the user’s account email to the network, 108 days after the earliest user report we found of the email appearing in its web requests (our piece).
Our Numbers
By our count, four of the eight studies, 50%, were run by the model’s maker on its own models, and four by outside groups. Of the four outside studies, one, METR’s, relied on data that Anthropic provided and partly redacted, and one, PSI’s, used a non-public Anthropic model. Two of the eight linked public code repositories in what we read: Snorkel’s experiment and PSI’s Lean formalisation. OpenAI’s is the only one of the eight that we reported as stating no human baseline.
Connecting the Coverage
Each study above links to our full piece, with its sources and caveats. The counts here are our reading of those eight pieces, not a ranking of the models or the studies.
The Structural Read
As we read the four studies run by model makers, each reports results on the maker’s own models: Anthropic’s EVE and memory work on Claude, OpenAI’s BioBasicsBench on its own models, and Google DeepMind’s RSA results from its own agents.
As we read the four outside studies, their distance from the labs varies: Visa and Snorkel AI ran their own tests, while METR relied on data that Anthropic gave it and partly redacted, and PSI used a non-public Anthropic model.
As we read the material each study linked, two of the eight point to public code: Snorkel linked a Git repo with a fully reproducible experiment, and PSI’s repository publishes its Lean 4 formalization.
Sam McCandlish, Anthropic co-founder, on X, 5 November 2026
“We redacted a lot, and I accept that this makes it harder for others to check METR’s work”
Three Implications
FOR READERS COMPARING SCORES The studies use different tasks, metrics and models, so a 66.7% on OpenAI’s lab benchmark and a 42.8% failure rate on Visa’s shopping benchmark measure different things.
FOR COMPANIES DEPLOYING AGENTS Visa says that in 54% of failed runs the AI still made a purchase, and Snorkel’s Chris Glaze says harness settings swung agent behaviour the most.
FOR CHECKING THE CLAIMS McCandlish wrote that Anthropic’s redactions make METR’s work harder for others to check, and PSI’s paper has not been through peer review.
The analysis says: “The surface, not the model, governs autonomy.”
As we read it, Snorkel’s finding that harness settings swung agent behaviour the most, more than the choice of model, sits close to that idea. The analysis does not discuss these studies.
What Is Not Established
The eight studies use different tasks, metrics and models, so no score here can be compared with another. Our counts depend on how we classified each study: we counted DeepMind’s RSA work as run by the model maker because DeepMind’s own agents did it, though outside cryptographers took part. Whether each study reported a human baseline is not something our coverage recorded for every one, so we do not count it beyond OpenAI’s stated absence.
This Week In The AI Supercycle
Anthropic’s Reality Check, The Capital Squeeze, & The New World Order
This week’s issue: for two years the AI build-out was priced as a growthstory. This week it started being priced as a credit story.
Eight AI agent studies came out on 5 and 6 November. Half were run by the companies whose models were being tested, half by outside groups, and two of the outside four depended on Anthropic’s data or model. The scores measure different things and are not comparable.
A note on sourcing. Every study and incident here is drawn from FourWeekMBA’s own earlier pieces, each linked in the text with its primary sources. The classification of who ran each study, the counts and the 50% figure are ours. The studies use different tasks and metrics, and their scores are not comparable.
Gennaro Cuofano is a CRO and tech executive who has worked in AI since late 2015, starting with bringing NLP, natural-language generation, voice and chatbot products to the marketing industry. His background is in law and finance: he holds a Master's degree in Law and an International MBA with an emphasis on corporate finance (LUISS Business School and the University of San Diego, 2012), and worked as a financial analyst at a real-estate investment firm in San Diego and as an assistant controller. His work focuses on business model strategy, business engineering and, more broadly, structural analysis: how companies make money, read from their own filings. He created FourWeekMBA and leads research there and at The Business Engineer, his newsletter on AI and business strategy, with over 95,000 subscribers and more than 1,000 published analyses. His writing has also appeared on Entrepreneur (for example, How to Digitize a Family Business in Europe) and HackerNoon. Find him on LinkedIn and Substack. See how we source.
![This Week In Business AI: The $195B Month [Week #9-2026]](https://i0.wp.com/fourweekmba.com/wp-content/plugins/contextual-related-posts/default.png?resize=150%2C150&ssl=1 "This Week In Business AI: The $195B Month [Week #9-2026]")
