OpenAI and Broadcom’s Jalapeño Chip Posts Strong Inference Numbers — Believe the Direction, Scrutinize the Scoreboard

OpenAI has released its own benchmark numbers for Jalapeño, the custom inference ASIC it built with Broadcom — the results are directionally credible and structurally important, but they are self-reported on open-weight models, and the comparison is an inference-only chip against a general-purpose GPU.

What Happened

OpenAI and Broadcom have put detailed numbers behind Jalapeño — the custom inference ASIC the two companies unveiled earlier in 2026. According to OpenAI’s own disclosure, the chip was run on InferenceX, SemiAnalysis’s public, open-source inference benchmark — a fixed model set spanning GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T — and the results were reported by OpenAI, not independently reproduced on the chip by a third party. That framing matters and belongs in the first sentence of any serious analysis of these numbers.

On that benchmark, OpenAI reports Jalapeño delivered 1.5 to 1.9 times more AI work per watt at peak throughput, 1.7 to 3.6 times lower end-to-end latency, and 2.1 to 4.1 times higher performance on highly interactive workloads — all measured against Nvidia’s GB200 and GB300 systems. The chip is rated at 700 watts but ran at or below 550 watts during the tested runs, which is a favorable framing. The target is approximately 50% lower inference cost than GPU-based serving. End-2026 is the deployment target, at initial volumes; the real production ramp is a 2027–2028 story. Generation 2 is deep in development; Generation 3 is taking shape.

InferenceX is a credible, neutral framework — SemiAnalysis’s benchmark, not an OpenAI invention. But OpenAI ran Jalapeño on it and reported the results itself, the comparison is an inference-only ASIC against general-purpose GPUs, and the model set is open-weight, meaning these numbers say nothing about how Jalapeño performs on OpenAI’s own frontier models. None of that makes the numbers false. It makes them self-reported on a credible framework, which is a different confidence level than an independently audited head-to-head.

The Structural Read

Start with the category. An ASIC is a chip built to do one thing; a GPU is built to do many. When you design straight at the memory bottleneck that limits GPU efficiency on LLM inference — and strip out everything the chip does not need for that one job — you are supposed to win on performance per watt for that job. This is precisely the story of Google’s TPU, now a decade old. The 1.5-to-1.9× per-watt figure, impressive as it is, is a feature of the category, not a miracle of engineering. Treating it as a shock requires forgetting what ASICs are for. The Beyond Nvidia’s Moat analysis at Business Engineer sets this up precisely: the ASIC-versus-GPU comparison on inference is apples-to-oranges in Jalapeño’s favor on this one axis, while Nvidia’s chips also train, flex across workloads, and carry a decade of software ecosystem that an ASIC does not inherit.

The benchmark framing compounds this. InferenceX is SemiAnalysis’s public, open-source benchmark — its model set is fixed (GPT-OSS 120B, DeepSeek R1 670B, Kimi K2.5 1T), and OpenAI did not choose those models. But OpenAI ran Jalapeño on that benchmark and reported the results itself, comparing against the best available GB200/GB300 numbers, without an independent party reproducing the Jalapeño run. That is self-reported on a neutral framework — directionally credible, not independently verified. The open-weight model set is also a meaningful gap: it says nothing about how Jalapeño performs on o3, o4-mini, or GPT-5 — the workloads that actually drive OpenAI’s revenue and cost structure. And the chip is not deployed yet. End-2026 is a target for initial volumes; production at scale, where power delivery, yield, networking, and software maturity all bite, is a 2027–2028 story.

None of those caveats make the numbers false, and they matter less than they might — because beating Nvidia on a benchmark was never the actual prize. OpenAI burns billions of dollars a quarter, and inference is its single largest cost. As The AI Value Chain framework makes clear, the lab that controls its own compute layer captures a structurally different margin profile than one that rents GPUs from Nvidia at whatever price Nvidia sets. Nvidia has been raising GPU prices and memory remains the binding constraint — a chip that serves OpenAI’s models at roughly half the cost transforms its unit economics whether or not every headline multiple holds under independent scrutiny.

Three Implications