Thinking Machines Lab, the AI lab led by Mira Murati, says it trained Inkling 2 Small with a native 4-bit forward pass on Nvidia GB300s, “from the start of pretraining through the final RL rollout”. In a technical post published on 13 November, it calls the model “the largest model we are aware of that was trained end-to-end with a native 4-bit forward pass”.
Update, 14 November 2026, 18:30 UTC: what is inside Inkling 2 Small. Thinking Machines’ model card describes Inkling 2 Small as a 48-layer decoder-only transformer with a sparse mixture-of-experts backbone that routes each token to 8 of 384 experts, with 594 billion parameters in total and 17 billion active, a context window of up to 1M tokens and an Apache 2.0 licence. The card says attention is “a hybrid of local and global layers”; the published configuration lists 40 sliding-window layers and 8 global layers, a 5:1 ratio, and groups the layers into attention-residual blocks of 6, or 8 blocks by our arithmetic. Only about 2.9% of the parameters are active per token, also our arithmetic. Machine-learning researcher Sebastian Raschka, who drew the architecture on X, said the model uses attention residuals like Kimi K3 “but without the linear-attention hybrid”; the configuration lists no linear-attention layers, while the diagram shows Kimi K3.1 Pro mixing Kimi Delta Attention with sparse attention layers.
The post’s own table shows Nvidia’s 4-bit format, NVFP4, at 15 petaflops on a GB300, against 5 for 8-bit MXFP8 and 2.5 for 16-bit BF16: 3x and 6x, by our arithmetic. The measured gain in its largest jobs is smaller: an 8% wall-clock speedup in pretraining and 20% in RL over full MXFP8 training, the lab says.
Business Pill · GPUs explained
An evergreen Business Engineer explainer on what GPUs do and why AI depends on them. It teaches the general idea only and does not discuss Thinking Machines or this post.
The key insight: As we read the post, the headline speed of the 4-bit format and the gain the lab measured are far apart: Nvidia’s NVFP4 runs up to 6x faster than BF16 on a GB300, by the lab’s table, yet its end-to-end recipe gave an 8% pretraining speedup in its largest jobs, with the lab pointing to a communications bottleneck.
The lab says it uses three numerical formats during training: BF16, MXFP8 and NVFP4. It says NVFP4 “is stored with 3.6x fewer bits than BF16, but it runs up to 6x faster on GB300s”, while it “greatly increases quantization error” compared with the other two.
For Inkling 2 Small, the post lists three choices: NVFP4 precision in the expert layers for forward passes, “bolstered by a dedicated kernel”; MXFP8 precision for backward passes, with re-quantization and stochastic rounding to reduce gradient bias; and quantization techniques for GPU communications.
The lab says it configures numerics the same way from the start of pretraining through the final RL rollout, which it says “eliminates the need for calibration when a quantized checkpoint is delivered at the end of training”.
GB300 petaflops by numerical format, from the table in Thinking Machines’ 13 November post; the 6x and 3x ratios are our arithmetic, and the measured end-to-end gains were 8% in pretraining and 20% in RL over full MXFP8.
Why the Gain Is 8%, Not 3x
The post explains the gap itself. It says that with the new NVFP4 matrix multiplications, the forward pass of its mixture-of-experts layers “should theoretically have gotten about three times faster” than its MXFP8 implementation, but that it then ran into “a bottleneck: communications during expert parallelism”.
The lab also says its first experiments with full NVFP4 training produced “very large hits” to its model’s compute efficiency, so it kept 4-bit to the forward pass. It keeps the last 16.6% of layers in higher precision, computed in MXFP8, against the 15% it says Nvidia’s recipe sets.
From Weeks to 68 Hours
The post says a training run of a model similar in size to Inkling 2 Small took several weeks months ago, and that “Inkling 2 Small was pretrained in just 68 hours on GB300 NVL72 racks”, or about 2.8 days by our arithmetic. Murati wrote on X that the 4-bit work was “one of the many improvements across the stack that took pretraining from weeks to 68 hours”.
As we read it, the larger change is the run time: the lab says a run of a similar-size model took several weeks months ago, and that Inkling 2 Small was pretrained in 68 hours, about 2.8 days by our arithmetic, with 4-bit described as one of many improvements.
As we read the recipe, 4-bit stays where the lab says it hurts accuracy least: the forward pass of expert layers, with backward passes in MXFP8 and the last 16.6% of layers kept in higher precision.
Thinking Machines Lab, End-to-End 4-Bit Training, 13 November 2026
“In our largest jobs, this recipe provides us with an 8% wall-clock speedup in pretraining and a 20% speedup in RL over full MXFP8 training.”
Three Implications
FOR AI LABS TRAINING ON GB300S The lab says it uses NVFP4 for expert-layer forward passes, MXFP8 for backward passes and quantized communications, configured the same way from pretraining through RL.
FOR NVIDIA The lab says NVFP4 runs up to 6x faster than BF16 on GB300s, and that it is exploring how its recipes will evolve on Vera Rubin systems.
FOR MODEL DELIVERY The lab says keeping the same numerics throughout removes the need for calibration when a quantized checkpoint is delivered.
The framework says: “Training was a cost center: episodic, concentrated among roughly twenty frontier model labs, amortizable across trillions of tokens.”
As we read it, Thinking Machines’ post is about cutting that training cost through faster arithmetic; the framework does not discuss Thinking Machines.
What Is Not Established
The post does not give Inkling 2 Small’s parameter count, the number of GPUs used, or the cost of the 68-hour run. It does not say how much of the cut from several weeks came from the 4-bit recipe and how much from other changes, and it does not quantify accuracy against an MXFP8 baseline beyond describing the effects.
The figures are the lab’s own and were not independently verified. Thinking Machines was not contacted for this piece.
This Week In The AI Supercycle
AI Lost Its Central Banker, the Lab Oligopoly, and the Future of Open Source
This week’s issue: the Nvidia leadership change, read as the AI build-out losing its central banker: who now finances the GPU landlords, backstops the data centres and sponsors open models, and why two labs are becoming the bank.
Thinking Machines says Nvidia’s 4-bit format runs up to 6x faster than BF16 on a GB300, yet its end-to-end 4-bit recipe delivered an 8% pretraining speedup and 20% in RL in its largest jobs, as part of the changes that it says took Inkling 2 Small’s pretraining to 68 hours.
A note on sourcing. The training details, format table and speedups come from Thinking Machines Lab’s Connectionism post “End-to-End 4-Bit Training” of 13 November 2026, read in full on 14 November; the Murati quote comes from her post on X the same day. Ratios and day conversions are our arithmetic. The lab was not contacted for this piece.
Gennaro Cuofano is a CRO and tech executive who has worked in AI since late 2015, starting with bringing NLP, natural-language generation, voice and chatbot products to the marketing industry. His background is in law and finance: he holds a Master's degree in Law and an International MBA with an emphasis on corporate finance (LUISS Business School and the University of San Diego, 2012), and worked as a financial analyst at a real-estate investment firm in San Diego and as an assistant controller. His work focuses on business model strategy, business engineering and, more broadly, structural analysis: how companies make money, read from their own filings. He created FourWeekMBA and leads research there and at The Business Engineer, his newsletter on AI and business strategy, with over 95,000 subscribers and more than 1,000 published analyses. His writing has also appeared on Entrepreneur (for example, How to Digitize a Family Business in Europe) and HackerNoon. Find him on LinkedIn and Substack. See how we source.
