Claude, GPT-6 Meet Under 55% of a Banker’s Deal Checklist

Julius Research, the lab behind the Y Combinator-backed data-analysis app Julius, released DeepFinance on Thursday, a benchmark of 21 private equity and investment banking tasks that it says would take a human professional 20 to 30 hours each. On its leaderboard, the best configuration, Anthropic’s Claude Opus 5.5, met 54.8% of the rubric criteria on average, and the top four, two Claude and two GPT models, scored between 52.9% and 54.8%.

Y Combinator chief executive Garry Tan shared the launch on X, calling it “the first long horizon benchmark for PE/IB work” and writing that “Solving high finance is on the path to AGI.” Julius is a Y Combinator company from the Summer 2022 batch, according to YC’s directory.

What the Benchmark Measures

• The tasks: 21 problems covering valuation, due diligence, underwriting and financial modelling, each paired with a “data room” of 19 files on average (spreadsheets, Word documents, emails, PDFs and CSV extracts), built from real deal data that was scrubbed and anonymised, Julius Research wrote.
• The time: an average of 27.1 hours of human work per task, according to Julius Research. Its chief executive’s launch post on X put the range at 20 to 40 hours; the blog and leaderboard say 20 to 30 hours.
• The grading: on average 84 rubric criteria per task, labelled as output, workflow or judgment, with partial credit for the right steps. The score is the share of criteria met, averaged over 10 attempts per task.
• The authors: the tasks were written and reviewed by professionals with backgrounds at firms such as J.P. Morgan, UBS and Evercore and institutions such as Harvard Business School, Wharton and Columbia Business School, according to Julius Research.
• The setup: each model ran in the harness it is most likely to be used in (Claude models in Claude Code, GPT models in Codex, Gemini, Muse, DeepSeek, GLM, Nemotron and Inkling models in the third-party opencode-ai tool, and Kimi, Qwen, Grok and Mistral in their own coding tools), with web search on and the ask-user tool off.

What Was Posted on X

Julius’s chief executive (bio: “CEO @JuliusAI”) announced DeepFinance on X at 22:41 UTC on 5 November, describing 21 problems built from real deal data and written by experts with backgrounds at Evercore, J.P. Morgan and UBS. The post puts the human time at 20 to 40 hours per task; Julius Research’s blog says 20 to 30 hours, averaging 27.1.

Y Combinator chief executive Garry Tan quoted the launch at 02:13 UTC on 6 November, calling it “the first long horizon benchmark for PE/IB work” and writing that “Solving high finance is on the path to AGI.” Julius is a Y Combinator company from the Summer 2022 batch.

What the Leaderboard Shows

• The top four, Claude Opus 5.5 (54.8%), Claude Sonnet 5.5 (54.6%), GPT-6 Astra (54.2%) and GPT-6.1 Sol (52.9%), are “not statistically distinguishable,” Julius Research wrote.
• The best model from another provider, Google’s Gemini 4 Argon, scored 41.3%. By our arithmetic, that is 13.5 points below the top score.
• Cost: GPT-6.1 Sol averaged $0.81 per task against $8.63 for Claude Opus 5.5. By our arithmetic, Opus cost about 10.7 times as much for 1.9 more points. GPT-6.1 Terra was the cheapest model above 30%, at $0.38 per task and 43.5%.
• Claude Fable 5.1 scored 45.6% at $12.02 per task, the most expensive model tested, and Julius Research wrote that it was “not worth the cost” for this work.
• The bottom group, Nemotron 3 Ultra, Inkling and Inkling Small, scored 10.9% to 14.5%. Thinking Machines’ Inkling 2 Small scored 29.8%.

Where the Agents Failed

Julius Research wrote that agents often find the relevant evidence without acting on it. In one acquisition task, moving the closing date lowered the required offer by $1.25 million, and “All attempts by frontier models missed this saving,” it wrote, even though several had identified the buyer’s right to choose the date.

It also wrote that models “often execute calculations correctly after selecting the wrong inputs”: in a lending task, agents calculated debt capacity correctly from an incorrect earnings subtotal.

Business Pill · Verification Cost

A short explainer on verification cost: what it takes to check work before you can rely on it. It teaches the general idea only; the figures in this piece come from the sources named in the text.

Connecting the Coverage

DeepFinance joins a run of new tests of AI agents on work that is closer to real jobs than older benchmarks: Only 81 of 522 Top AI Benchmarks Measure Organizational Work, Mercor’s study of what benchmarks leave out; Claude Agents Read the Grader in 60% of Default Snorkel Runs, on how harness settings change agent behaviour; Visa: AI Shopping Agents Fail 42.8% of Runs, Often Still Buy; and OpenAI Leads Claude by 264 Elo in a $20K AI Deception Game, Dwarkesh Patel’s test of social deception, where OpenAI’s models led.

The Structural Read

As we read the leaderboard, the frontier models finish about half of an expert’s checklist on deal work, and the price of the last few points is steep.

As we read Julius Research’s failure notes, the gap it describes is judgment, not arithmetic: agents found the evidence and did the maths, but did not always act on what the evidence implied.

Julius Research grades the path as well as the answer, which it argues is how investment committees judge analysts’ work.

Three Implications

FOR BANKS AND PE FIRMS Julius Research says even frontier models are weak at turning evidence into judgment, such as reconciling a CFO’s view with an analyst’s.

FOR MODEL BUYERS On this test, GPT-6.1 Sol came within 1.9 points of the top score at about a tenth of Opus’s cost per task, by our arithmetic.

FOR BENCHMARK MAKERS Julius Research argues that grading only the final number misses the assumptions behind it, and grades workflow and judgment too.

The Business Engineer Lens

This story maps onto the Business Engineer framework The Product Overhang Doctrine.

The framework says: “The second curve is product capability — what users of your specific product can actually do, end-to-end, in production.”

As we read it, DeepFinance tries to measure that second curve for deal work: whether an agent can carry an assignment end to end, not whether it can answer one question. The framework itself does not discuss DeepFinance or Julius.

What Is Not Established

DeepFinance is a new benchmark from a company that sells AI data-analysis software, and its tasks, data rooms and rubrics are not public on its site; the scores have not been reproduced independently. The rubric score is the share of checklist items met, not a pass rate for a whole deal, and Julius Research notes that the number of failed checks should not be read as the number of independent reasoning failures. Models ran in different harnesses, several sharing opencode-ai, so the scores measure a model and its tool together.

The Bottom Line

On Julius Research’s new DeepFinance benchmark, the best AI agents met just under 55% of the checklist an expert would apply to 21 private equity and investment banking assignments, with Claude and GPT models tied at the top. By our arithmetic, the 21 tasks add up to about 569 hours of analyst work.

95,000+ executives read Business Engineer for the AI strategy frameworks cited by ChatGPT, Claude, and Perplexity.

A note on sourcing. The benchmark design, scores, costs and failure examples are from Julius Research’s DeepFinance blog post and leaderboard, published 5 November 2026. Garry Tan’s comment and the 20-to-40-hour range are from posts on X by Tan and by Julius’s chief executive. Julius’s Y Combinator batch is from YC’s company directory. The 13.5-point gap, the 10.7x cost ratio and the 569 hours are our arithmetic.