Fifty professional analysts working with Claude Code went with their agent’s lean 86% of the time on four analytical choices, Anthropic said in a research post published on 27 October 2026. The lean was set by coin flip and given to the agent in hidden instructions the analysts were not told about. If the agent’s input made no difference, Anthropic says, “we’d expect about 50%”.
That is 36 percentage points above the no-influence baseline, by our arithmetic. Anthropic also found that 77% of analysts repeated their agent’s argument when explaining a choice, while only 15% credited the agent anywhere.
Business Engineer Course · The FDE Playbook
An evergreen Business Engineer course on forward-deployed engineering, how AI gets put to work inside organisations. It teaches the general idea only and does not discuss Anthropic’s study.
The key insight: In Anthropic’s study of 50 professional analysts, choices matched a coin-flip lean placed in their Claude Code agent 86% of the time, 36 points above the about-50% no-influence baseline by our arithmetic, and 77% repeated the agent’s argument while 15% credited it.
Anthropic recruited 61 analysts with causal-inference experience for a randomized controlled trial; 11 chose not to use the agent, leaving 50 in the main analysis. Their task was to advise a fictional farming cooperative on whether a paid advisory program worked and should be expanded. They used Claude Code running Claude Opus 4.8 at High effort.
Each analyst faced four analytical decisions with two reasonable options, such as measuring productivity per hectare or per unit of operating cost, and then a final recommendation. The agent was told to hold its assigned lean as its own view, to change its mind if the evidence turned decisively against it, and never to misrepresent results.
Share of 50 analysts whose choice matched their Claude Code agent’s coin-flip lean on each decision, from Anthropic’s 27 October 2026 research post. Anthropic says about 50% would be expected if the agent made no difference.
What the Analysts Did
Matches with the agent’s lean ran at 92% for the outcome measure, 82% for the comparison method, 81% for which farms the estimate covered and 90% for weighting, Anthropic reports. On the final recommendation, 54% of analysts recommended rolling out the program when the agent leaned that way, against 36% when it leaned the other way, a gap of 18 points by our arithmetic.
In 38 cases where an analyst had stated a preference before the agent weighed in, they switched 60% of the time when the agent disagreed and 6% when it agreed. Anthropic calls that a 54-point gap but says the estimate is imprecise, with a 95% confidence interval from 24 to 82 points.
Asked afterwards, 52% of analysts rated the final recommendation as more their own than their agent’s. Anthropic says both groups were roughly equally influenced. Experience and expertise did not predict who deferred, according to the study.
The Other Side of the Ledger
Of the 11 participants who worked without the agent, six, or 55%, made serious methodological errors, against three of the 50 who used it, or 6%, Anthropic reports. The no-agent group also took a median 90 minutes against 65. By our arithmetic, that is about 9 times the error rate and about 28% less time with the agent.
Anthropic says the two groups were not directly comparable: those who skipped the agent were less experienced on average. It writes that this “isn’t a simple cautionary tale against working with AI”.
Why It Matters for AI-Assisted Work
Anthropic writes that if experts defer to agents without realizing it, “the agents’ preferences—which could be arbitrary or misaligned—could shape important decisions about policy proposals or research directions without anyone having chosen those preferences”. It adds that when separate teams defer to models with the same preferences, their agreement “may no longer be independent”.
As we read the study, the influence ran through the analytical choices rather than the final verdict: the final-recommendation gap was 18 points by our arithmetic, but the four leans alone predicted 82% of recommendations, Anthropic reports.
As we read the credit figures, the gap between repeating the agent’s argument (77%) and crediting it (15%) is 62 points by our arithmetic.
As we read Anthropic’s caveats, the leans were deliberately strengthened and the study covered one task, one field and one model, so the 86% describes that set-up.
Anthropic, How AI agents shape experts’ judgment calls, 27 October 2026
“a human reviewer is only an independent check if their judgment is their own”
Three Implications
FOR TEAMS USING AGENTS Anthropic reports that 52% of analysts rated the final recommendation as more their own than their agent’s, while both groups were roughly equally influenced.
FOR RESEARCH AND POLICY WORK Anthropic writes that if separate teams defer to models with the same preferences, their agreement may no longer be independent.
FOR QUALITY Anthropic reports serious errors in 6% of analyses with the agent and 55% without, while saying the two groups were not directly comparable.
The framework says “Judgment becomes the highest-value layer of work.”
The framework is the owner’s argument about agent harnesses, not evidence about this study; it does not discuss Anthropic’s analyst experiment or its figures.
What Is Not Established
Anthropic lists the limits itself. It deliberately strengthened the agents’ lean, studied one task in one field with one model, and reports averages across analysts. It notes that real agents do lean on their own: with no lean assigned, Claude Opus 4.8 made the same five choices in 112 of 123 runs. Because the study strengthened the leans on purpose, Anthropic says the effect “might be smaller” when agents are not given clear preferences and instructions to voice them. The study does not measure how strongly analysts held their own views.
This Week In The AI Supercycle
Execution Fault Lines, Concentrated Demand, and Geopolitical Firewalls
This week’s issue: the AI build is not short of demand for its parts. It is short of cheap money, and its end revenue rests on a narrow base of large, often venture-funded buyers.
In Anthropic’s study, expert analysts followed a coin-flip lean placed in their Claude Code agent 86% of the time, 36 points above the 50% no-influence baseline by our arithmetic, and most repeated the agent’s reasoning without crediting it. Analysts who used the agent made fewer serious errors and finished faster, though Anthropic says it cannot be sure the agent caused the difference because the groups differed.
A note on sourcing. All study figures come from Anthropic’s research post of 27 October 2026, which reports on its own paper. Figures labelled our arithmetic are our own calculations.
Gennaro Cuofano is a CRO and tech executive who has worked in AI since late 2015, starting with bringing NLP, natural-language generation, voice and chatbot products to the marketing industry. His background is in law and finance: he holds a Master's degree in Law and an International MBA with an emphasis on corporate finance (LUISS Business School and the University of San Diego, 2012), and worked as a financial analyst at a real-estate investment firm in San Diego and as an assistant controller. His work focuses on business model strategy, business engineering and, more broadly, structural analysis: how companies make money, read from their own filings. He created FourWeekMBA and leads research there and at The Business Engineer, his newsletter on AI and business strategy, with over 95,000 subscribers and more than 1,000 published analyses. His writing has also appeared on Entrepreneur (for example, How to Digitize a Family Business in Europe) and HackerNoon. Find him on LinkedIn and Substack. See how we source.
