Siemens’ Fuse EDA team raised the mean composite verification score of a chip-design agent running Claude Opus 4.8 from approximately 55% to 95%, without changing the model, according to an NVIDIA technical blog published on 14 October. The team ran the evaluations in NVIDIA’s open NeMo Gym library.
The blog says agent turns fell from 155 to 53 and output tokens from approximately 81K to 29K. By our arithmetic, the agent delivered about 4.8 times as much verified score per 1,000 output tokens after the changes. NVIDIA’s AI account highlighted the results on X on 30 October.
Short explainer · Siemens Chip Agent Went From 55% to 95% on the Same Claude Model
A vertical news Short narrated from this piece’s headline and opening line.
Business Pill · AI AGENTS
A short Business Engineer explainer on AI agents. It teaches the general idea only and does not discuss the Siemens or NVIDIA work.
The key insight: NVIDIA’s blog says the Siemens team held Claude Opus 4.8 fixed and refined the agent’s characterization skills and MCP tool descriptions, and the composite verification score rose from approximately 55% to 95% while output tokens fell from about 81K to 29K; by our arithmetic that is about 4.8 times the verified score per 1,000 output tokens.
The blog describes standard-cell characterization as “the process of generating accurate timing, power, and noise models for logic cells across operating conditions”. It says the work is required whenever a semiconductor foundry issues a new process design kit.
The Solido Characterization Suite agent orchestrates Siemens’ Solido tools through the Siemens Fuse EDA AI Agent, according to the blog. It says the agent must handle tool setup, large-scale SPICE simulation, distributed compute, Liberty file generation, debugging and final validation.
The blog names the core problem: “Errors often produce plausible outputs instead of clean failures”. It says an undetected error early in setup can silently propagate through downstream stages.
Composite verification score per 1,000 output tokens for the Siemens chip-characterization agent running Claude Opus 4.8: about 0.68 before tuning (~55%, ~81K tokens) and about 3.28 after (~95%, ~29K tokens), from NVIDIA’s 14 October blog; the ratios are our arithmetic.
What Was Posted on X
How do you make a chip design agent more reliable and efficient
The @Siemens EDA team used NVIDIA NeMo Gym to check the engineering files its agent produced, trace errors, and improve the agentโs skills and tool descriptions.
In its standard cell characterization benchmark, those changes raised the composite verification score from ~55% to ~95%, with 66% fewer agent turns and 64% fewer output tokens.
We just published a new techblog that walks through what the team measured, what they changed, and how other engineering teams can apply the approach โ https://nvda.ws/4Beujcb
NVIDIA’s AI account highlighted the Siemens results on X on 30 October, describing a rise in the composite verification score from ~55% to ~95% with 66% fewer agent turns and 64% fewer output tokens, and linking to an NVIDIA technical blog on the work.
Same Model, Better Harness
The blog says the team conducted approximately 25 rollouts “using Claude Opus 4.8 running in Claude Code as the fixed model and agent harness”. Each run took approximately 1.5 to 2 hours on two cells across 36 PVT corners, a subset of the 229-cell GF180MCU dataset.
Across rounds, the team refined the agent’s characterization skills and MCP tool descriptions, the blog says. To improve tool selection, it says they “simplified overlapping file and shell interfaces and defined tool inputs with explicit JSON schemas”.
The blog reports the result in one table: the composite score rose from approximately 55% to 95%, turns fell from 155 to 53, a reduction of approximately 66%, and output tokens fell from approximately 81K to 29K, a reduction of approximately 64%.
Siemens chip-characterization agent before and after tuning: turns 155 to 53, output tokens about 81K to 29K, as reported in NVIDIA’s 14 October blog (development benchmark, about 25 rollouts).
How the Verifier Scores Runs
NeMo Gym, the blog says, “defines the task, tools, state, and verifier for each rollout” and records thinking traces, messages and tool actions. Its composite reward combines coverage, validity, relative validation, accuracy and integrity, and the blog says “The reward is zero if any gate fails”.
The blog says end-to-end evaluation found failures that intermediate checks missed, such as missing cells and incorrect slew thresholds. Its conclusion: “engineering agents must be evaluated against final artifacts rather than individual tool calls”.
By our arithmetic, the score per 1,000 output tokens went from about 0.68 (55 divided by 81) to about 3.28 (95 divided by 29). About 52K fewer output tokens were used per run, which is a cost line wherever output tokens are billed.
The Structural Read
As we read the blog, the measurable gain came from the layer around the model: it says the team “simplified overlapping file and shell interfaces and defined tool inputs with explicit JSON schemas”.
As we read the reward design, the verifier is strict by construction: the blog says “The reward is zero if any gate fails”, across coverage, validity, relative validation, accuracy and integrity.
As we read the efficiency numbers, fewer turns and fewer output tokens per run reduce the compute each verified result consumes; the blog reports turns down about 66% and output tokens down about 64%.
NVIDIA Technical Blog, 14 October 2026
“engineering agents must be evaluated against final artifacts rather than individual tool calls”
Three Implications
FOR ENTERPRISE AGENT BUILDERS The blog lists the method: evaluate final artifacts, write a domain-specific verifier, collect many rollouts, and refine skills and MCP tool interfaces.
FOR MODEL PROVIDERS The blog says the agent ran Claude Opus 4.8 in Claude Code as the fixed model and harness; the gains it reports came from changes outside the model.
FOR NVIDIA The blog presents NeMo Gym as an open library for evaluating and improving models and agents using environments, here applied to an EDA workflow.
The analysis puts it in one line: “The next professional jump is not a better prompt.”
As we read it, the Siemens result fits that framing: the model was fixed, the team refined skills and tool interfaces, and a verifier measured each run. The framework was published in April 2026 and does not discuss this case.
What Is Not Established
The blog reports a development benchmark on two cells of an open-source process kit, not a production deployment. The documents we read give no dollar cost per run, no customer names and no results for the full 229-cell library.
The scores are from NVIDIA’s blog about a collaboration with Siemens; we did not rerun them. The blog says the process was “driven by a coding agent with a human in the loop”, and it does not separate how much of the gain came from skills versus tool descriptions.
This Week In The AI Supercycle
Execution Fault Lines, Concentrated Demand, and Geopolitical Firewalls
This week’s issue: the AI build is not short of demand for its parts. It is short of cheap money, and its end revenue rests on a narrow base of large, often venture-funded buyers.
Siemens’ Fuse EDA team took a Claude Opus 4.8 chip-characterization agent from approximately 55% to 95% on its composite verification score by rewriting skills and tool descriptions in NVIDIA NeMo Gym, with the model held fixed, according to NVIDIA’s 14 October blog. Turns fell from 155 to 53 and output tokens from about 81K to 29K; by our arithmetic that is about 4.8 times the verified score per 1,000 output tokens.
A note on sourcing. Findings come from NVIDIA’s technical blog of 14 October 2026, read in full on 30 October 2026, and NVIDIA AI’s post on X; we did not contact NVIDIA, Siemens or Anthropic and did not rerun the benchmark. Figures labelled our arithmetic are our own.
Gennaro Cuofano is a CRO and tech executive who has worked in AI since late 2015, starting with bringing NLP, natural-language generation, voice and chatbot products to the marketing industry. His background is in law and finance: he holds a Master's degree in Law and an International MBA with an emphasis on corporate finance (LUISS Business School and the University of San Diego, 2012), and worked as a financial analyst at a real-estate investment firm in San Diego and as an assistant controller. His work focuses on business model strategy, business engineering and, more broadly, structural analysis: how companies make money, read from their own filings. He created FourWeekMBA and leads research there and at The Business Engineer, his newsletter on AI and business strategy, with over 95,000 subscribers and more than 1,000 published analyses. His writing has also appeared on Entrepreneur (for example, How to Digitize a Family Business in Europe) and HackerNoon. Find him on LinkedIn and Substack. See how we source.
