Opus 5.5 Tops Cognition’s Test; GPT-6.1 Sol Costs 12x Less

Cognition, which builds the Devin coding agent, released FrontierCode 2.0 on 27 October 2026, a coding benchmark graded on whether open-source maintainers would merge the model’s work. On its leaderboard, Claude Opus 5.5 scores highest at 45.5%, at a mean cost of $12.82 per task, while OpenAI’s GPT-6.1 Sol scores 39.0% at $1.03.

By our arithmetic, GPT-6.1 Sol reaches 86% of Opus 5.5’s score at 8% of its cost per task; put the other way, Opus 5.5 costs 12.4 times as much for 6.5 more points.

Short explainer · Cognition’s coding test, cost vs score

A vertical news Short narrated from the published text of this piece and related FourWeekMBA pieces.

Business Pill · The model and the harness

An evergreen Business Engineer explainer on why products built on the same model differ, because of the harness around it. It explains the general idea and does not discuss Cognition or FrontierCode.

The Price of the Last Points

Cognition’s leaderboard lists each model at its best-scoring reasoning effort, and says cost “is the mean USD spend per rollout at list API pricing, including public discounts.” Behind Opus 5.5, GPT-6 Astra scores 42.8% at $6.68 per task and Claude Sonnet 5.5 scores 41.0% at $19.94, the highest cost among the top three.

At the low end, Claude Haiku 5.5 scores 27.7% at $0.37 per task and GPT-6 Luna scores 21.9% at $0.17. In its post on X, Cognition said “GPT-6.1 Sol leads in cost efficiency, scoring 39% at $1.03 per task.” The post gave Opus 5.5 and GPT-6 Astra average costs of $12.8 and $6.67; the leaderboard lists $12.82 and $6.68.

In its blog post, Cognition wrote: “The Pareto frontier includes Opus 5.5 on the high end along with Luna, Haiku, and Sol on the low-cost end, with GPT-6.1 Sol offering particularly strong performance relative to its cost.”

What Was Posted on X

Cognition’s own post sorts the models into groups: Opus 5.5 and GPT-6 Astra lead on score, GPT-6.1 Sol leads on cost efficiency at 39% for $1.03 per task, and GPT-6 Luna and Haiku 5.5 lead on overall cost. It says the Chinese open-source models it names, Kimi K3 and GLM 5.3, score 17%, and the American open-source models Beam and Inkling 3%.

The @scaling01 post says the top 20 on FrontierCode 2 are all OpenAI and Anthropic models plus Grok 4.7 xhigh at #20, and puts DeepSeek V4.1 Flash at rank 52, adding a sarcastic line about China catching up. Those ranks are the poster’s; the leaderboard table we read lists one row per model at its best effort, where Grok 4.7 is 7th and DeepSeek V4.1 Flash 16th.

Open Models Score Lower, and Not Always Cheaper

Cognition’s post said “Chinese OSS models lag far behind, with Kimi K3 and GLM 5.3 scoring at 17%.” On the leaderboard, Kimi K3 scores 17.7% at $4.47 per task and GLM 5.3 scores 17.8% at $7.52.

By our arithmetic, Kimi K3 costs 4.3 times GPT-6.1 Sol per task and GLM 5.3 costs 7.3 times, at less than half its score. DeepSeek V4.1 Flash scores 19.3% at $0.38 per task, about Haiku 5.5’s cost, 8.4 points lower by our arithmetic.

The American open-source models Cognition named score lower still: Beam 1.0 at 2.9% and Inkling Small at 3.2%. Cognition’s blog says: “Our results also highlight significant room for improvement for open source models, even more so for western OSS models.”

A Harder Test Widened the Gap

FrontierCode 2.0 is a new 132-task suite, according to the leaderboard. Cognition says its tasks are drawn from 55% more repositories and include 43% more blocking criteria, and that, compared with FrontierCode 1.1, reference solutions modify 67% more files and 89% more lines of code, in codebases that are 25% larger.

Every model Cognition compared scored lower on the new version. Opus 5.5 fell from 54.6% on FrontierCode 1.1 to 45.5%, GPT-6 Astra from 53.3% to 42.8%, Grok 4.7 from 47.6% to 28.6%, DeepSeek V4.1 Flash from 43.9% to 19.3% and Kimi K3 from 44.2% to 17.7%.

By our arithmetic, Opus 5.5’s lead over Kimi K3 grew from 10.4 points to 27.8 points. Cognition wrote that FrontierCode 2.0 “separates models more clearly across tiers of capability,” adding: “The benchmark is far from saturated: the top-performing model, Opus 5.5, scores only 45%.”

Thinking Longer Can Lower the Score

Cognition says FrontierCode is one of the few coding benchmarks where writing additional correct code does not necessarily raise a score, and that “models can score lower at higher reasoning effort.” It loosened its scope criteria in version 2.0: the failure rate on scope criteria, averaged across effort levels, is 3.8%, compared with 9.0% in FrontierCode 1.1.

In results pooled from four models, Cognition says blocking criteria in its “thoroughness” category had the highest failure rate at every effort level, above core behavior. In one example on the leaderboard page, Opus 5.5 fixes a reported bug in Mock Service Worker but skips the user’s cleanup code, and fails at every effort level.

Connecting the Coverage

On 26 October we reported OpenAI says GPT-6.1 Terra matches Astra at a tenth the price, and on 19 October OpenAI’s COBOL test: GPT-6.1 Sol scores 86.6%, up from 5.4%. On 23 October we reported Moonshot prices Kimi K3.1 Pro at half of Claude Opus 5.5.

Today we reported Claude Opus 5 tops a new XR coding benchmark at 50.9%. On 24 October we reported Anthropic token prices fell about 50% in a year, data show, and on 1 October that Cognition and Factory bet opposite sides of the AI stack. What’s new: a cost-per-task view across 25 models on one mergeability test.

The Structural Read

As we read it, the leaderboard measures cost per task, not per token: Cognition defines cost as the mean USD spend per rollout at list API pricing, so it reflects both a model’s prices and how much work it does on each task.

As we read it, cheaper is not the same as open on this test: Kimi K3 and GLM 5.3 cost $4.47 and $7.52 per task on Cognition’s leaderboard, more than GPT-6.1 Sol’s $1.03, at under half its score.

As we read it, the harder test separated the field: Opus 5.5’s lead over Kimi K3 grew from 10.4 points on FrontierCode 1.1 to 27.8 points on 2.0, by our arithmetic from Cognition’s table.

Three Implications

FOR TEAMS BUYING CODING AGENTS Cognition’s leaderboard shows a 12.4x cost-per-task gap between the top score and GPT-6.1 Sol’s, by our arithmetic, for 6.5 points of score.

FOR OPEN-MODEL VENDORS Cognition’s blog says its results highlight significant room for improvement for open source models, and more so for western ones.

FOR BENCHMARK READERS Cognition owns FrontierCode and builds SWE-2, which scores 23.8% on it; its costs are at list API pricing.

The Business Engineer Lens

This story maps onto the Business Engineer framework Tokenomics: The Economics of AI.

The framework puts it this way: the token “is simultaneously the unit of cognition the model produces, the unit of compute the data center serves, the unit of price the lab charges, and the unit of value the enterprise extracts.”

As we read it, Cognition’s cost-per-task figures sit where the last two meet: what a lab charges for its tokens, summed over one piece of finished work. The framework was written in May 2026 and does not discuss FrontierCode or Cognition.

What Is Not Established

FrontierCode 2.0 is Cognition’s own benchmark, and Cognition also builds coding models that appear on it: its SWE-2 scores 23.8% at $1.26 per task. The tasks were built by more than 20 open-source developers, according to Cognition, and graded with rubrics that Cognition calls subjective and reviews with a quality-control pipeline.

Costs are at list API pricing, not what enterprise customers pay under negotiated terms, and runs used different agent harnesses, including Claude Code, Codex and Devin CLI. The results cover 132 tasks, and scores at a model’s other reasoning efforts differ from the best-effort figures shown here.

The Bottom Line

On Cognition’s new mergeability benchmark, Opus 5.5 scores highest at $12.82 a task, GPT-6.1 Sol scores 39.0% at $1.03, and the Chinese open-source models Cognition named score under 18% at $4.47 and $7.52.