Baseten says its open-source inference engine for DeepSeek-V4.1-Flash, written from scratch in Rust and CUDA, decodes about 2,060 tokens per second on eight B300 GPUs and 1,740 on four GB300s. By our arithmetic, that is about 257 tokens per second per GPU on eight chips against 435 on four: twice the GPUs bought about 18% more speed on a single request.
The company posted the figures on X at 22:43 UTC on 13 November and published the code as the GitHub repository deepseek-ultrafast. Its README also reports prefilling 1,000,000 tokens in 11.7 seconds on eight B300s, about 85,000 tokens per second.
Business Pill · Inference
An evergreen Business Engineer explainer on inference, the cost of running a trained AI model for every request. It teaches the general idea only and does not discuss Baseten or DeepSeek.
The key insight: As we read the README, the headline speed is a single-request number: by our arithmetic, eight B300s deliver about 257 tokens per second each against 435 for four GB300s, so the extra four GPUs bought about 18% more speed.
2,060 Tokens a Second, Measured on One Prompt
The README gives a decode step time of 2.71 milliseconds on eight B300s and 3.30 milliseconds on four GB300s, with an acceptance length of 5.6 tokens per step. “Each decode step drafts a block of tokens and verifies it; the acceptance length is the average number of tokens a step emits.”, it explains.
The speeds are measured on one coding prompt that asks the model for a Tetris game in HTML, with thinking disabled in the reproduction request. Baseten says it sees 1,980 to 2,170 tokens per second on eight B300s and 1,700 to 1,770 on four GB300s when the demo is rerun.
Phil Howes, whose X profile says he is building Baseten, wrote in the post Baseten quoted that “the core engine is just rust + cuda. no pytorch, no sglang, no dsls, etc.” The README says the kernels were optimized by coding agents that could read the source of FlashInfer and SGLang while they worked.
The engine also shares each conversation’s KV cache between servers over RDMA. “Thanks to DeepSeek’s extreme KV cache compression, the transfers are very fast and can be done on every request.”, the README says, adding that the setup “removes the need for prefix-aware routing and enables better load balancing”.

What Speed Costs on the Open Market
Baseten sells DeepSeek V4.1 Flash Fast through its Model APIs at $0.60 per million input tokens and $2.40 per million output tokens, according to its pricing page, twice the $0.30 and $1.20 it charges for the standard version. The README says the Fast model is available on Baseten Model APIs; it does not say which engine serves it.
Artificial Analysis measures hosted endpoints of DeepSeek V4.1 Flash 1017 (Max) as five-day averages. On its page read on 16 November, Baseten’s FAST endpoint ran at 842.9 output tokens per second against 161.6 for its standard endpoint. By our arithmetic, that is 5.2 times the speed for 2 times the output price.
The same page shows LithosAI’s ULTRA CHAT endpoint at 819.7 tokens per second against 208.6 for its standard one, at $1.40 against $0.60 per million output tokens, and Fireworks’ ULTRA FAST endpoint at 722.4 against 231.1, at $9.60 against $1.20. By our arithmetic, Fireworks charges 8 times the output price for 3.1 times the speed, and LithosAI 2.3 times for 3.9 times.
Artificial Analysis lists Baseten (FAST) as the fastest provider by time per Intelligence Index task, at 2.7 minutes, and DeepInfra as the cheapest, at $0.29 per million output tokens with an output speed of 116.7 tokens per second. It says prices vary up to 28.8x across providers. DeepSeek’s own API shows 173.6 tokens per second there.
Baseten’s 10 November blog post on the Fast variant said “The model is on Artificial Analysis at 991 output tokens/s and the lowest time per task”. The five-day average we read six days later was 842.9.
Why a Flash Model Gets Its Own Engine
Baseten argues that “with the end of tokenmaxxing leaderboards and cost per task now central to every enterprise’s evaluation of agents and models, flash variants are the balanced choice for quality and economics”. The same post describes the DeepSeek model as a 552B-parameter MoE with 8B parameters active during prefill and 16B during decode.
The blog post adds that on Flash models with fewer active parameters on fast hardware, the Python scheduler in SGLang constrains throughput. Howes’s post says the new engine uses no SGLang.
Speed and price have run through our recent coverage: DeepSeek rationing its free chat, Tencent’s Hy4 Flash task costs, Codex capacity errors and Nvidia Rubin’s DeepSeek results.

What an Outside Reader Found in the Code
In an X article on 16 November, the engineer who posts as @athletic_coder, who says he works on post-training at Zomato, wrote that TP4 and TP8 mean “GPUs cooperating on one request, rather than four or eight independent copies each producing that rate”.
He also wrote that the README’s million-token prefill is a separate result and “does not establish that the Tetris decode rate was achieved with a million tokens already in context”. Those readings are his; the figures he cites match the README.
The Structural Read
As we read Artificial Analysis, speed is already priced as its own product line: Baseten’s FAST endpoint runs 5.2 times faster than its standard one for 2 times the output price, by our arithmetic.
As we read Baseten’s blog post, the pitch is cost per task rather than leaderboard scores: it says flash variants are the balanced choice for quality and economics now that cost per task is central to enterprise evaluations.
Phil Howes, on X, 13 November 2026
“we built an inference engine for deepseek v4.1-flash from scratch.”
Three Implications
FOR BUYERS OF INFERENCE Artificial Analysis lists Baseten (FAST) at 2.7 minutes per Intelligence Index task, the fastest, and DeepInfra at $0.29 per million output tokens, the cheapest.
FOR GPU OPERATORS The README reports 1,740 tokens per second on four GB300s and 2,060 on eight B300s for one request, and a 1,000,000-token prefill in 11.7 seconds.
FOR OPEN-SOURCE USERS The engine is on GitHub and needs four GB300s, four or eight B300s, or four or eight B200s, plus the CUDA 13 toolkit and a Redis instance, the README says.
The Business Engineer Lens
This story maps onto the Business Engineer framework The State of the Inference Economy.
The framework’s thesis: “Inference is a revenue engine: continuous, distributed across millions of applications and agents, incurring marginal cost with every query”.
As we read it, Baseten’s two price lines put a number on that marginal cost: the FAST endpoint costs $2.40 per million output tokens against $1.20 for the standard one. The framework predates the engine and does not discuss Baseten.
What Is Not Established
The README’s rates come from a single request on one prompt; Baseten did not publish throughput for many users served at once, a cost per token for the engine or a power figure. Artificial Analysis tests hosted endpoints on its own workloads, so its 842.9 tokens per second is not like for like with the README’s 2,060.
Baseten’s X post lists the four-GPU setup as 4xB300, while the README table says four GB300s; the README lists both chips as sm_103a. We did not run the code, and Baseten was not contacted for this piece.
The Bottom Line
Baseten’s open-source engine decodes DeepSeek-V4.1-Flash at about 2,060 tokens per second on eight B300s in its own test, but by our arithmetic each GPU delivers about 257 tokens per second against 435 on four GB300s. On Artificial Analysis, Baseten’s fast endpoint runs 5.2 times faster than its standard one for 2 times the output price.
95,000+ executives read Business Engineer for the AI strategy frameworks cited by ChatGPT, Claude, and Perplexity.
A note on sourcing. We read in full Baseten’s X post of 13 November 2026 and the Phil Howes post it quoted, the README of the basetenlabs/deepseek-ultrafast repository on GitHub, Baseten’s 10 November blog post and its pricing page, and the Artificial Analysis provider page for DeepSeek V4.1 Flash 1017 (read 16 November). The @athletic_coder X article is secondary analysis and is attributed to its author.
Per-GPU rates, speed and price multiples are our own arithmetic. We did not run the code. Baseten was not contacted for this piece.
Sources: basetenlabs/deepseek-ultrafast README (GitHub) · Baseten on X (13 Nov 2026) · Baseten blog: optimizing DeepSeek V4.1 Flash 1017 (10 Nov 2026) · Baseten pricing · Artificial Analysis: DeepSeek V4.1 Flash 1017 providers (read 16 Nov 2026) · @athletic_coder X article (16 Nov 2026)









