Baseten Runs DeepSeek V4.1 Flash at 2,060 Tokens/s on B300s

Baseten says its open-source inference engine for DeepSeek-V4.1-Flash, written from scratch in Rust and CUDA, decodes about 2,060 tokens per second on eight B300 GPUs and 1,740 on four GB300s. By our arithmetic, that is about 257 tokens per second per GPU on eight chips against 435 on four: twice the GPUs bought about 18% more speed on a single request.

The company posted the figures on X at 22:43 UTC on 13 November and published the code as the GitHub repository deepseek-ultrafast. Its README also reports prefilling 1,000,000 tokens in 11.7 seconds on eight B300s, about 85,000 tokens per second.

Business Pill · Inference

An evergreen Business Engineer explainer on inference, the cost of running a trained AI model for every request. It teaches the general idea only and does not discuss Baseten or DeepSeek.

The key insight: As we read the README, the headline speed is a single-request number: by our arithmetic, eight B300s deliver about 257 tokens per second each against 435 for four GB300s, so the extra four GPUs bought about 18% more speed.

2,060 Tokens a Second, Measured on One Prompt

The README gives a decode step time of 2.71 milliseconds on eight B300s and 3.30 milliseconds on four GB300s, with an acceptance length of 5.6 tokens per step. “Each decode step drafts a block of tokens and verifies it; the acceptance length is the average number of tokens a step emits.”, it explains.

The speeds are measured on one coding prompt that asks the model for a Tetris game in HTML, with thinking disabled in the reproduction request. Baseten says it sees 1,980 to 2,170 tokens per second on eight B300s and 1,700 to 1,770 on four GB300s when the demo is rerun.

Phil Howes, whose X profile says he is building Baseten, wrote in the post Baseten quoted that “the core engine is just rust + cuda. no pytorch, no sglang, no dsls, etc.” The README says the kernels were optimized by coding agents that could read the source of FlashInfer and SGLang while they worked.

The engine also shares each conversation’s KV cache between servers over RDMA. “Thanks to DeepSeek’s extreme KV cache compression, the transfers are very fast and can be done on every request.”, the README says, adding that the setup “removes the need for prefix-aware routing and enables better load balancing”.

On Artificial Analysis, fast endpoints of DeepSeek V4.1 Flash 1017 run 3.1 to 5.2 times faster than the same p
On Artificial Analysis, fast endpoints of DeepSeek V4.1 Flash 1017 run 3.1 to 5.2 times faster than the same provider’s standard endpoint, at 2 to 8 times the output price (our arithmetic). Data: Artificial Analysis five-day averages and list prices, read 16 November 2026.

What Speed Costs on the Open Market

Baseten sells DeepSeek V4.1 Flash Fast through its Model APIs at $0.60 per million input tokens and $2.40 per million output tokens, according to its pricing page, twice the $0.30 and $1.20 it charges for the standard version. The README says the Fast model is available on Baseten Model APIs; it does not say which engine serves it.

Artificial Analysis measures hosted endpoints of DeepSeek V4.1 Flash 1017 (Max) as five-day averages. On its page read on 16 November, Baseten’s FAST endpoint ran at 842.9 output tokens per second against 161.6 for its standard endpoint. By our arithmetic, that is 5.2 times the speed for 2 times the output price.

The same page shows LithosAI’s ULTRA CHAT endpoint at 819.7 tokens per second against 208.6 for its standard one, at $1.40 against $0.60 per million output tokens, and Fireworks’ ULTRA FAST endpoint at 722.4 against 231.1, at $9.60 against $1.20. By our arithmetic, Fireworks charges 8 times the output price for 3.1 times the speed, and LithosAI 2.3 times for 3.9 times.

Artificial Analysis lists Baseten (FAST) as the fastest provider by time per Intelligence Index task, at 2.7 minutes, and DeepInfra as the cheapest, at $0.29 per million output tokens with an output speed of 116.7 tokens per second. It says prices vary up to 28.8x across providers. DeepSeek’s own API shows 173.6 tokens per second there.

Baseten’s 10 November blog post on the Fast variant said “The model is on Artificial Analysis at 991 output tokens/s and the lowest time per task”. The five-day average we read six days later was 842.9.

Why a Flash Model Gets Its Own Engine

Baseten argues that “with the end of tokenmaxxing leaderboards and cost per task now central to every enterprise’s evaluation of agents and models, flash variants are the balanced choice for quality and economics”. The same post describes the DeepSeek model as a 552B-parameter MoE with 8B parameters active during prefill and 16B during decode.

The blog post adds that on Flash models with fewer active parameters on fast hardware, the Python scheduler in SGLang constrains throughput. Howes’s post says the new engine uses no SGLang.

Speed and price have run through our recent coverage: DeepSeek rationing its free chat, Tencent’s Hy4 Flash task costs, Codex capacity errors and Nvidia Rubin’s DeepSeek results.

Decode tokens per second per GPU in Baseten’s open-source DeepSeek engine: 435 on four GB300s (1,740 in total) and about 257 on eight B300s (2,060 in total)
Twice the GPUs bought about 18% more decode speed on one request. Source: basetenlabs/deepseek-ultrafast README on GitHub; per-GPU rates are our arithmetic.

What an Outside Reader Found in the Code

In an X article on 16 November, the engineer who posts as @athletic_coder, who says he works on post-training at Zomato, wrote that TP4 and TP8 mean “GPUs cooperating on one request, rather than four or eight independent copies each producing that rate”.

He also wrote that the README’s million-token prefill is a separate result and “does not establish that the Tetris decode rate was achieved with a million tokens already in context”. Those readings are his; the figures he cites match the README.

The Structural Read

As we read Artificial Analysis, speed is already priced as its own product line: Baseten’s FAST endpoint runs 5.2 times faster than its standard one for 2 times the output price, by our arithmetic.

As we read Baseten’s blog post, the pitch is cost per task rather than leaderboard scores: it says flash variants are the balanced choice for quality and economics now that cost per task is central to enterprise evaluations.

Phil Howes, on X, 13 November 2026

“we built an inference engine for deepseek v4.1-flash from scratch.”

Three Implications

FOR BUYERS OF INFERENCE Artificial Analysis lists Baseten (FAST) at 2.7 minutes per Intelligence Index task, the fastest, and DeepInfra at $0.29 per million output tokens, the cheapest.

FOR GPU OPERATORS The README reports 1,740 tokens per second on four GB300s and 2,060 on eight B300s for one request, and a 1,000,000-token prefill in 11.7 seconds.

FOR OPEN-SOURCE USERS The engine is on GitHub and needs four GB300s, four or eight B300s, or four or eight B200s, plus the CUDA 13 toolkit and a Redis instance, the README says.

The Business Engineer Lens

This story maps onto the Business Engineer framework The State of the Inference Economy.

The framework’s thesis: “Inference is a revenue engine: continuous, distributed across millions of applications and agents, incurring marginal cost with every query”.

As we read it, Baseten’s two price lines put a number on that marginal cost: the FAST endpoint costs $2.40 per million output tokens against $1.20 for the standard one. The framework predates the engine and does not discuss Baseten.

What Is Not Established

The README’s rates come from a single request on one prompt; Baseten did not publish throughput for many users served at once, a cost per token for the engine or a power figure. Artificial Analysis tests hosted endpoints on its own workloads, so its 842.9 tokens per second is not like for like with the README’s 2,060.

Baseten’s X post lists the four-GPU setup as 4xB300, while the README table says four GB300s; the README lists both chips as sm_103a. We did not run the code, and Baseten was not contacted for this piece.

This Week In The AI Supercycle

AI Lost Its Central Banker, the Lab Oligopoly, and the Future of Open Source

This week’s issue: the Nvidia leadership change, read as the AI build-out losing its central banker: who now finances the GPU landlords, backstops the data centres and sponsors open models, and why two labs are becoming the bank.

Read the full issue →

Business Engineer Framework

The State of the Inference Economy

The Business Engineer framework on how inference became the revenue engine of the AI economy.

Read the Map of AI →

The Bottom Line

Baseten’s open-source engine decodes DeepSeek-V4.1-Flash at about 2,060 tokens per second on eight B300s in its own test, but by our arithmetic each GPU delivers about 257 tokens per second against 435 on four GB300s. On Artificial Analysis, Baseten’s fast endpoint runs 5.2 times faster than its standard one for 2 times the output price.

95,000+ executives read Business Engineer for the AI strategy frameworks cited by ChatGPT, Claude, and Perplexity.

A note on sourcing. We read in full Baseten’s X post of 13 November 2026 and the Phil Howes post it quoted, the README of the basetenlabs/deepseek-ultrafast repository on GitHub, Baseten’s 10 November blog post and its pricing page, and the Artificial Analysis provider page for DeepSeek V4.1 Flash 1017 (read 16 November). The @athletic_coder X article is secondary analysis and is attributed to its author.

Per-GPU rates, speed and price multiples are our own arithmetic. We did not run the code. Baseten was not contacted for this piece.

Sources: basetenlabs/deepseek-ultrafast README (GitHub) · Baseten on X (13 Nov 2026) · Baseten blog: optimizing DeepSeek V4.1 Flash 1017 (10 Nov 2026) · Baseten pricing · Artificial Analysis: DeepSeek V4.1 Flash 1017 providers (read 16 Nov 2026) · @athletic_coder X article (16 Nov 2026)

Scroll to Top
FourWeekMBA