Nvidia’s Vera Rubin NVL72 produced 9.11 times as many tokens per dollar as the GB300 NVL72 at 200 tokens per second per user, and 20.9 times as many at 325, on a 744-billion-parameter agentic model. The figures are from an official preview of SemiAnalysis’s InferenceX benchmark, which the firm posted on X on 1 November.
SemiAnalysis prices a Rubin chip at $3.61 an hour against $2.31 for a GB300, about 56% more. By our arithmetic, a 9.11x lead per dollar at that price gap means roughly 14 times the tokens per chip at 200 tokens per second per user, and about 33 times at 325.
Business Pill · The Memory Wall
A short explainer on the memory wall in AI computing. It teaches the general idea only and says nothing about this benchmark.
The key insight: As we read SemiAnalysis’s preview, Rubin’s lead over GB300 per dollar is widest at the fast, per-user speeds that agentic workloads need, not at the slow, bulk end of the curve.
SemiAnalysis wrote that “HBM CHIPS CAN REACH OVER 1000 tok/s/user on giant 700B+ models via agent swarm-written software, according to a new NVIDIA AgentX submission”. It added that on Rubin NVL72 “it can achieve 20.9x better perf per TCO than GB300 NVL72 at even “ultrafast” speeds of 325 tok/s/user and up to 9.11x better perf per TCO at 200 tok/s/user”.
The attached chart is titled “GLM5.2/GLM5.3 744B Agentic Total Tokens per $1 TCO vs. P90 Interactivity”. It compares “Vera Rubin NVL72 FP4 (Dynamo TRTLLM)” with “GB300 NVL72 FP4 (Dynamo TRTLLM)”, at the cost tier “Owning at Large Hyperscaler Volume”, updated 30 October.
The Rubin result carries a label: “InferenceX Official Preview: Results may change as validation and publication continue.” On the chart, the Rubin curve peaks near 150 million tokens per dollar at about 200 tokens per second per user and runs past 1,000. The GB300 curve stops at about 325.
SemiAnalysis InferenceX official preview, GLM-5.3 744B agentic (updated 30 October 2026): Vera Rubin NVL72 delivered 9.11x GB300 NVL72’s tokens per $1 TCO at 200 tok/s/user and 20.9x at 325, both FP4 on Dynamo TRTLLM; TCO per chip-hour $3.61 (VR200) vs $2.31 (GB300). Preview results may change.
What Was Posted on X
BREAKING NEWS๐จ: HBM CHIPS CAN REACH OVER 1000 tok/s/user on giant 700B+ models via agent swarm-written software, according to a new NVIDIA AgentX submission.
Furthermore, on Rubin NVL72, it can achieve 20.9x better perf per TCO than GB300 NVL72 at even “ultrafast” speeds of 325 tok/s/user and up to 9.11x better perf per TCO at 200 tok/s/user. (1/2)๐งต
SemiAnalysis posted these figures on 1 November: on Rubin NVL72, “20.9x better perf per TCO than GB300 NVL72” at 325 tok/s/user and “up to 9.11x” at 200 tok/s/user, from an InferenceX preview on GLM-5.3 that it labels as subject to change.
Where the Gain Comes From
In a second post, SemiAnalysis said the Rubin submission uses a new “Mooncake Store feature of Extended GPU Memory, where KVCache stored on Vera’s LPDDR5X is pooled across all 18 Rubin compute trays using NVLink instead of the previous method of using RDMA, resulting in greater than 10x better bandwidth”.
As we read it, the gain is not only faster chips. It also comes from software: the post credits “agent swarm-written software” in Nvidia’s submission, and a memory change that moves the KV cache across the rack over NVLink rather than RDMA.
Huang Said 30x on Stage
At GTC Berlin on 21 October, we reported, Jensen Huang said Vera Rubin is another 30 times beyond Blackwell, while a slide behind him showed “30X” in tokens per dollar on a SemiAnalysis chart. The new preview, on GLM-5.3, shows 9.11x and 20.9x against GB300; our 21 October report does not name the model behind the 30x slide.
Speed has changed the multiple before: on 2 October we reported Rubin’s 67x gain shrinking to 1.4x to 3x at 60 to 100 tokens per second. Here the multiple grows with speed, from 9.11x at 200 to 20.9x at 325. As we read the chart, that is because GB300’s tokens per dollar fall steeply as it nears its top speed, while Rubin’s curve keeps going.
The Business Lever: Cost per Token at Agent Speeds
As we read it, the gap matters most for fast per-user workloads such as agents: in this preview it is larger at 325 tokens per second per user than at 200. A buyer comparing the two is weighing a chip that is 56% dearer per hour, on SemiAnalysis’s figures, against many times the output per dollar at those speeds.
As we read the chart, Rubin’s tokens per dollar grow from 9.11x to 20.9x GB300’s as the speed per user rises from 200 to 325 tokens per second.
By our arithmetic, SemiAnalysis’s hourly costs of $3.61 for a Rubin chip and $2.31 for a GB300 mean Rubin is about 56% dearer per chip-hour, so the per-chip output gap is larger than the per-dollar gap.
As we read SemiAnalysis’s second post, part of the gain is memory design: pooling the KV cache across the rack over NVLink instead of RDMA, which it says gives greater than 10x better bandwidth.
SemiAnalysis, on X, 1 November 2026
“on Rubin NVL72, it can achieve 20.9x better perf per TCO than GB300 NVL72”
Three Implications
FOR NVIDIA’S CUSTOMERS SemiAnalysis prices Rubin at $3.61 per chip-hour against $2.31 for GB300, and its preview shows 9.11x to 20.9x the tokens per dollar on GLM-5.3.
FOR AMD AND OTHER CHIPMAKERS The InferenceX page lists TCO per chip-hour for AMD’s MI325X ($1.1) and MI355X ($1.5) alongside Nvidia’s parts, so per-dollar comparisons depend on each chip’s measured throughput.
FOR MODEL PROVIDERS SemiAnalysis’s post says HBM chips can reach over 1,000 tokens per second per user on 700B-plus models, which sets the speed range the comparison covers.
The framework says: “Inference accounts for 80 to 90 percent of the lifetime cost of a production AI system.”
As we read it, a benchmark of tokens per dollar is a measure of that lifetime cost. The framework predates this preview and does not discuss Rubin or GB300.
The Rubin result is labelled a preview that may change. The figures cover one model (GLM-5.3, 744 billion parameters) on one agentic scenario, priced at SemiAnalysis’s own TCO tier. The InferenceX page notes that disaggregated setups such as Dynamo TRTLLM count cost per decode or prefill chip rather than per total chip count. The roughly 14x and 33x per-chip multiples are our arithmetic from the stated ratios and hourly costs. SemiAnalysis does not describe the agent-written software beyond the phrase quoted.
This Week In The AI Supercycle
Anthropic’s Reality Check, The Capital Squeeze, & The New World Order
This week’s issue: for two years the AI build-out was priced as a growthstory. This week it started being priced as a credit story.
In SemiAnalysis’s InferenceX preview, Nvidia’s Vera Rubin NVL72 delivered 9.11 times GB300’s tokens per dollar at 200 tokens per second per user and 20.9 times at 325, on GLM-5.3. Rubin costs about 56% more per chip-hour on SemiAnalysis’s figures, so by our arithmetic the output per chip is roughly 14x to 33x.
A note on sourcing. The multiples, chart and hourly costs come from SemiAnalysis’s two posts on X on 1 November and its InferenceX GLM-5.3 page (updated 30 October), where the Rubin result is labelled a preview that may change. Chart readings are approximate. We did not contact Nvidia or SemiAnalysis. The per-chip multiples are our arithmetic.
Gennaro Cuofano is a CRO and tech executive who has worked in AI since late 2015, starting with bringing NLP, natural-language generation, voice and chatbot products to the marketing industry. His background is in law and finance: he holds a Master's degree in Law and an International MBA with an emphasis on corporate finance (LUISS Business School and the University of San Diego, 2012), and worked as a financial analyst at a real-estate investment firm in San Diego and as an assistant controller. His work focuses on business model strategy, business engineering and, more broadly, structural analysis: how companies make money, read from their own filings. He created FourWeekMBA and leads research there and at The Business Engineer, his newsletter on AI and business strategy, with over 95,000 subscribers and more than 1,000 published analyses. His writing has also appeared on Entrepreneur (for example, How to Digitize a Family Business in Europe) and HackerNoon. Find him on LinkedIn and Substack. See how we source.
