Ramp has published Ramp SWE-Bench v2, a private coding benchmark of 500 tasks built from production work by more than 200 of its engineers (Ramp on X). It runs every model twice: in its lab’s own agent harness and in a bash-only one.
The harness gap is largest outside the top tier. GPT-6.1 Terra and GPT-6 Luna each score 9.8 points higher in OpenAI’s Codex than in the bash-only harness, by our arithmetic from Ramp’s tables.
Business Pill · Vertical Integration
An evergreen Business Engineer explainer on vertical integration. It teaches the general idea only and does not discuss Ramp.
The key insight: On Ramp’s production-grounded benchmark, the agent harness adds up to 9.8 points for models outside the top four but under 1.5 for the leaders, by our arithmetic from Ramp’s tables.
How Ramp Tests
Tasks come from pull requests that Ramp’s in-house coding agent, Inspect, merged into production after engineer review. Each result is a single attempt at high reasoning effort, with web tools disabled.
The “Native” view runs each model in its lab’s harness, such as Claude Code or Codex. The “Bash” view uses mini-swe-agent, “a minimal harness where the only tool is bash”, per Ramp.

What Was Posted on X
RampLabs announced Ramp SWE-Bench v2 on X on 18 November: 500 tasks built from work shipped to production by over 200 of its engineers, with frontier models in their native harnesses. The scores in this piece come from the benchmark site it links.
Veeral Patel wrote on X that we can finally answer how much the harness matters without asking someone who is selling one. We quote his reading as his view.
Who Leads
In native harnesses, GPT-6.2 Astra in Codex leads at 76.4%. Google’s Carbon in OpenCode follows at 75.4%, then Claude Fable 5.2 in Claude Code and GPT-6.2 Sol in Codex at 73.4% each.
Where The Harness Matters
For the top models the harness barely moves the score: Fable 5.2 is 0.4 points higher native, Astra 1.4 and Carbon 1.4, by our arithmetic.
Models further down the table gain more. Grok 4.7 is 9.0 points higher in Grok Build, DeepSeek V4.1 Flash 7.4 points higher in its harness, and Claude Sonnet 5.5 5.6 points higher in Claude Code.
Two models do better without their native harness: Claude Opus 5.5 scores 3.4 points higher in the bash-only setup, and Muse Ember 2.6 points higher.
What It Costs
The harness also changes the bill. Astra’s mean cost per task is $3.31 natively against $1.03 bash-only, about 3.2 times, for 1.4 more points, by our arithmetic.
Fable 5.2 runs the other way: $3.79 a task in Claude Code against $6.45 bash-only, with near-identical scores.
Why It Matters
Patel’s reading, on X: “Finally we can answer how much the harness matters without asking someone who’s selling you one” (Veeral Patel). Other agent benchmarks we covered this week include Vals AI’s Lean proof test and OpenAI’s own research-agent data.
Coding agents are also becoming a distribution channel, as with Snap opening Lens building to Claude Code.
The Structural Read
As we read Ramp’s tables, the harness matters most for models outside the top tier, though two of them score higher without it.
As we read the cost columns, the harness is also a pricing decision: the same model can cost three times as much per task.
Three Implications
FOR ENGINEERING TEAMS On Ramp’s tables, the harness choice moves some models by up to 9.8 points.
FOR MODEL LABS The harness can be worth up to 9.8 points on Ramp’s tasks, by our arithmetic.
FOR BUYERS Astra costs about 3.2 times more per task natively than bash-only on Ramp’s tables.
The Business Engineer Lens
This story maps onto the Business Engineer framework The Agentic Harness War.
The analysis puts it this way: “Foundation models become relatively commoditized as orchestration layers capture differentiation.”
As we read Ramp’s tables, the harness gap is largest outside the top tier. The analysis predates Ramp’s benchmark and does not discuss it.
What Is Not Established
The tasks are private Ramp code, so results may not transfer to other codebases. Grading uses a panel of LLM judges, per Ramp. Ramp’s tables show results for 14 models in native harnesses.
The Bottom Line
On Ramp’s 500 production-grounded tasks, the native harness adds up to 9.8 points for models outside the top four but under 1.5 for the leaders, by our arithmetic from Ramp’s tables.
97,000+ executives read Business Engineer for the AI strategy frameworks cited by ChatGPT, Claude, and Perplexity.
A note on sourcing. Scores, costs and methodology are from Ramp’s SWE-Bench v2 site (native and bash views, last updated 17 November), read 19 November. The harness gaps, cost ratio and rankings are our arithmetic from Ramp’s tables. Patel’s comment is from his post on X.
Sources: Ramp SWE-Bench v2 site: native and bash-harness summary tables and methodology, read 19 Nov 2026 · RampLabs on X, 18 Nov 2026 · Veeral Patel on X, 18 Nov 2026









