Nvidia’s Vera Rubin NVL72 rack served DeepSeek-V4.1-Flash at 58 times the throughput of a GB300 NVL72 rack when each user gets 387 tokens per second, the open-source vLLM project said on X on 12 November. At peak throughput the gain is 2.7 times: 2.52 million tokens per second per GPU. The 58x figure holds at one speed setting, not across the board; it is about 21 times the peak multiple, by our arithmetic.
The results come from vLLM’s blog post on the SemiAnalysis AgentX benchmark, a workload built from real agentic coding traces. The work was led by Inferact, vLLM says, and SemiAnalysis, which built AgentX, praised it in its own post.
Business Pill · Tokens Explained
An evergreen Business Engineer explainer on tokens, the unit AI reads, writes and bills in. It teaches the general idea only and does not discuss the vLLM results.
The key insight: vLLM reported Nvidia’s Vera Rubin NVL72 at 58 times GB300 NVL72’s throughput on DeepSeek-V4.1-Flash when each user gets 387 tokens per second, and 2.7 times at peak throughput; by our arithmetic the headline multiple is about 21 times the peak one.
A serving benchmark traces a curve. Pack more users onto a rack and total throughput rises, but each user’s stream gets slower. SemiAnalysis writes that “it is important to consider both tokens per second per user (TPS – also known as interactivity) and TTFT, since these often come at the expense of one another.”
vLLM reported three points on Rubin’s curve: 2.52 million tokens per second per GPU at peak throughput (2.7 times GB300), 1.7 million at a P90 speed of 387 tokens per second per user (58 times GB300), and 1,810 tokens per second for a single session using tensor parallelism.
By our arithmetic from vLLM’s rounded multiples, GB300 delivers about 0.93 million tokens per second per GPU at its peak and about 29,000 at 387 tokens per second per user. Rubin keeps about 67% of its peak throughput at that speed; GB300 keeps about 3%. That is where the 58x comes from: the older rack is near the edge of its curve at that per-user speed.
One more detail matters for reading the 1.7 million. vLLM’s chart note says: “Throughput counts cached input, uncached input and output tokens”. In agentic coding most input is a cached prefix, so the figure is not a count of generated tokens.
Throughput per GPU on DeepSeek-V4.1-Flash, SemiAnalysis AgentX, at two points of the curve. Rubin figures and the 2.7x and 58x multiples are from vLLM (12 November 2026); the GB300 values are our arithmetic from those rounded multiples and are approximate.
What Was Posted on X
We optimized DeepSeek-V4.1-Flash on vLLM for Vera Rubin NVL72 and reached 58× the throughput of GB300 NVL72 at 1.7 million tokens/s/GPU and 387 tokens/s/user on SemiAnalysis AgentX.
Across the curve: 🔹 2.52 million tokens/s/GPU at peak throughput (2.7× GB300) 🔹 1.7 million tokens/s/GPU at P90 387 tokens/s/user (58× GB300) 🔹 1,810 tokens/s at one concurrent session with TP
The open-source vLLM project posted this at 22:31 UTC on 12 November, the first of a four-post thread. It gives the three points on Rubin’s curve: 2.52 million tokens per second per GPU at peak (2.7x GB300), 1.7 million at 387 tokens per second per user (58x GB300) and 1,810 tokens per second for one session. This piece takes every Rubin figure from this thread and the linked blog post.
SemiAnalysis, which built the AgentX benchmark the results were measured on, quoted the thread 24 minutes later, crediting Inferact and vLLM for the Rubin-optimized DeepSeek-V4.1-Flash. The post is praise only and adds no figures.
What vLLM Changed for Rubin
vLLM says it reworked its MoE, attention, GEMM, mHC and indexer kernels for Rubin, keeping bandwidth-bound data in each SM’s own memory domain and using wider MMA instructions in Rubin’s SM107 architecture. Prefill steps got about 28% faster and decode steps about 25% faster, according to the post.
The thread adds that “Rubin computes fast enough that a rack serves far more sessions at once”. vLLM pooled the rack’s memory, with one Mooncake KV pool across the DRAM and spare HBM of every tray. It says the prefix hit rate stays at 96.5–97% even at very high concurrency, and that with Dynamo’s router, P90 time to first token drops from 0.98 to 0.38 seconds; the blog says that figure comes from a 12-instance simulation.
How AgentX Measures
vLLM’s AgentX explainer describes the traces: a median 43 turns per session, a median input of 142,000 tokens, a median output of 444 tokens and a prefix-cache hit rate above 96%. SemiAnalysis says its benchmark configs mainly track public vLLM recipes and SGLang cookbooks on upstream images.
The Rubin numbers are vLLM’s own measurement on the AgentX workload, and the post links SemiAnalysis’s InferenceX site as the source of the GB300 comparison. vLLM thanks Nvidia for early access to Vera Rubin NVL72 hardware.
As we read it, the 58x is a statement about where each rack sits on its own curve. By our arithmetic Rubin keeps about 67% of its peak throughput at 387 tokens per second per user, while GB300 keeps about 3%. For an agent product that needs fast responses, that per-user speed is the relevant point; for batch work where latency does not matter, the 2.7x peak figure is the closer comparison.
vLLM on X, 12 November 2026
“Rubin computes fast enough that a rack serves far more sessions at once, and their live prefixes outgrow any per-rank cache, so we pooled the rack’s memory”
Three Implications
FOR INFERENCE PROVIDERS vLLM reports Rubin at 1.7 million tokens per second per GPU at 387 tokens per second per user, against about 29,000 implied for GB300 by our arithmetic.
FOR AGENT BUILDERS SemiAnalysis says per-user speed and time to first token often come at the expense of one another; vLLM says P90 time to first token fell from 0.98 to 0.38 seconds in its simulation.
FOR NVIDIA vLLM thanks Nvidia for early access to Vera Rubin NVL72, and its results are the second vLLM Rubin benchmark post after the 9 October preview.
The analysis says: “Inference is a revenue engine: continuous, distributed across millions of applications and agents, incurring marginal cost with every query.”
As we read it, the vLLM results show why the operating point matters: an agentic session served at high per-user speed is where Rubin’s reported gain is largest. The analysis predates the vLLM post and does not discuss Rubin.
Throughput at 387 tokens per second per user as a share of each rack’s own peak. Our arithmetic from vLLM’s 12 November figures and rounded multiples (Rubin 1.7M / 2.52M; GB300 implied), approximate.
What Is Not Established
The GB300 figures above are implied from vLLM’s rounded multiples, not read off the chart, so they are approximate. These are vLLM’s results on one model and one benchmark; we did not see an independent rerun. The posts give no prices, so the multiples say nothing directly about cost per token. Some of the memory pooling is described by vLLM as still experimental.
This Week In The AI Supercycle
The Great Convergence of Capital, Power, and Geopolitics
This week’s issue: for two years the AI build-out was priced as a supply story, and last week as a credit story. This week it was priced as a permission story, and the answer arrived as a cost, not a ban.
vLLM reported Rubin at 58 times GB300’s throughput on DeepSeek-V4.1-Flash at 387 tokens per second per user, and 2.7 times at peak throughput. By our arithmetic the headline multiple is about 21 times the peak one, because Rubin keeps about 67% of its peak throughput at that per-user speed while GB300 keeps about 3%.
A note on sourcing. We read the vLLM project’s four-post thread on X and its 12 November blog post in full, plus SemiAnalysis’s post quoting the thread, which was passed to us. Methodology details come from vLLM’s AgentX post and SemiAnalysis’s AgentX newsletter. The Rubin figures are vLLM’s own measurements; we did not rerun them. The 21x gap, the implied GB300 figures and the 67% and 3% shares are our arithmetic from vLLM’s rounded multiples. We did not contact vLLM, Inferact, SemiAnalysis or Nvidia.
Gennaro Cuofano is a CRO and tech executive who has worked in AI since late 2015, starting with bringing NLP, natural-language generation, voice and chatbot products to the marketing industry. His background is in law and finance: he holds a Master's degree in Law and an International MBA with an emphasis on corporate finance (LUISS Business School and the University of San Diego, 2012), and worked as a financial analyst at a real-estate investment firm in San Diego and as an assistant controller. His work focuses on business model strategy, business engineering and, more broadly, structural analysis: how companies make money, read from their own filings. He created FourWeekMBA and leads research there and at The Business Engineer, his newsletter on AI and business strategy, with over 95,000 subscribers and more than 1,000 published analyses. His writing has also appeared on Entrepreneur (for example, How to Digitize a Family Business in Europe) and HackerNoon. Find him on LinkedIn and Substack. See how we source.
