Nvidia researchers have published an open midtraining recipe for agentic coding models, and say it lifted Nemotron 3.5 Lightning’s SWE-bench Verified score from 60.7 to 64.5 after identical fine-tuning, according to a paper posted on arXiv on 15 October 2026.
The paper, “Priming Base Models for Agentic Post-Training: An Open Midtraining Recipe”, describes a training stage that sits between pretraining and post-training. Its authors write that frontier models now include such a stage, “but little has been published about how this stage is built.”
Business Pill · PRETRAINING EXPLAINED
A Business Engineer explainer on how a model learns before post-training. It explains the concept in general and says nothing about Nvidia’s paper.
The key insight: As we read it, Nvidia is publishing how to build a stage that its authors say frontier labs already use but rarely document: the paper sets out the data blend, the 256K-context training setup and the evaluation method, and tests them on Nvidia’s own Nemotron models.
According to the paper, the midtraining stage trains at 256K context for hundreds of billions of tokens on natural software-engineering data, such as packed repositories, pull requests, commit histories and build logs, together with a small share of agentic traces.
On Nemotron 3.5 Lightning, the authors report that 500B midtraining tokens improved SWE-bench Verified from 60.7 to 64.5, SWE-bench Multilingual from 55.8 to 62.8 and SWE-bench Pro from 32.0 to 35.5, after identical full supervised fine-tuning.
On the larger Nemotron 3.5 Super, the paper says 280B midtraining tokens improved DeepSWE, which it describes as a harder benchmark of long-horizon tasks, from 11.7 to 15.7, and SWE-bench Verified from 72.9 to 74.3.
Points gained on Nemotron 3.5 Lightning after 500B midtraining tokens, with identical full fine-tuning, as stated in the Nvidia researchers’ paper (arXiv 2610.22019, 15 October 2026): 3.8 on SWE-bench Verified, 7.0 on SWE-bench Multilingual, 3.5 on SWE-bench Pro and 1.0 on Terminal-Bench. The authors’ own results.
What Was Posted on X
Great paper from NVIDIA sharing an open midtraining recipe for agentic coding.
NVIDIA kept SFT identical and added one midtraining stage. SWE-bench Multilingual went from 55.8 to 62.8 on Nemotron 3.5 Lightning.
The stage sits between pretraining and post-training. It trains at 256K context on 500B tokens of natural software-engineering data, such as packed repositories, pull requests, commit histories and build logs, plus a small share of agent traces.
On the same model, SWE-bench Verified rose from 60.7 to 64.5 and SWE-bench Pro from 32.0 to 35.5. On the larger Nemotron 3.5 Super, DeepSWE rose from 11.7 to 15.7.
They also explain how they evaluated the recipe. Base-model proxy metrics did not predict post-training results, so every data choice was scored with short fixed SFT runs.
If you train coding agents, the recipe and its ablations are published.
DAIR.AI, which curates AI research papers, highlighted the Nvidia paper on 17 October, summarising the added midtraining stage at 256K context on 500B tokens of software-engineering data and the gains it reports on Nemotron 3.5 Lightning and Super. The figures are the paper’s own.
What Is in the Recipe
The paper says the agentic-trace corpus in the blend contains about 39B tokens, with over one million trajectories and 30M tool calls. It says filtering removes shortcuts such as solution leakage through Git history or internet lookup, unfinished trajectories and repetitive file-inspection loops.
The authors also report that midtraining made the models more robust across agent harnesses. On Nemotron 3.5 Super after 65B tokens and short fine-tuning, they put the average gain across harnesses at 8.6 points on SWE-bench Verified and 6.8 on Terminal-Bench.
They add that base-model proxy metrics such as bits-per-byte and LoCoDiff “did not reliably predict post-training results”, so they evaluate the recipe with short, fixed fine-tuning runs instead.
As we read it, the gain comes from data agents actually work with: the paper says most of the blend is natural software-engineering data such as repositories, pull requests, commit histories and build logs, with a small share of agent traces.
As we read it, the paper also makes a point about measurement: its authors say base-model proxy metrics did not reliably predict post-training results, so they judge the recipe with short, fixed fine-tuning runs.
As we read it, robustness across harnesses is part of the pitch for open models: the paper says midtraining improved every harness it tested, with an average 8.6-point gain on SWE-bench Verified on Nemotron 3.5 Super after 65B tokens and short fine-tuning.
Nvidia researchers, arXiv 2610.22019
“Frontier models, general-purpose and domain-specific alike, now include a midtraining stage between pretraining and post-training, but little has been published about how this stage is built.”
Three Implications
TEAMS TRAINING OPEN CODING MODELS The paper documents a midtraining blend and setup: 256K context, hundreds of billions of tokens of software-engineering data and a small share of agent traces.
EVALUATION LEADS The authors say bits-per-byte and LoCoDiff did not reliably predict post-training results, and evaluate with short fixed fine-tuning runs instead.
BUILDERS CHOOSING A HARNESS The paper reports that midtraining improved results on every agent harness the authors tested, on both Nemotron models.
The framework states the pattern as “Close the SCARCE layer (keep proprietary) + Open the ABUNDANT layer (commoditize)”.
As we read it, Nvidia is opening a training recipe, the know-how of building coding models, while the compute it runs on is what Nvidia sells. The framework was published in March 2026 and does not discuss this paper.
What Is Not Established
The benchmark scores are the authors’ own and were run on Nvidia’s Nemotron 3.5 models; the paper does not show the recipe applied to other companies’ models.
We did not reproduce the training runs, and we did not contact Nvidia.
Nvidia researchers have published an open midtraining recipe for agentic coding and report that 500B midtraining tokens lifted Nemotron 3.5 Lightning’s SWE-bench Verified score from 60.7 to 64.5 after identical fine-tuning.
The paper documents the data blend, a 256K-context training stage between pretraining and post-training, and an evaluation method built on short fine-tuning runs.
A note on sourcing. This piece is based on the Nvidia researchers’ paper on arXiv (2610.22019, submitted on 15 October 2026), read in full on 17 October 2026, and the DAIR.AI post on X that pointed to it. All benchmark figures are the authors’ own. We did not reproduce the training runs, and we did not contact Nvidia. The references to our earlier reporting link to our own pieces.
Gennaro Cuofano is a CRO and tech executive who has worked in AI since late 2015, starting with bringing NLP, natural-language generation, voice and chatbot products to the marketing industry. His background is in law and finance: he holds a Master's degree in Law and an International MBA with an emphasis on corporate finance (LUISS Business School and the University of San Diego, 2012), and worked as a financial analyst at a real-estate investment firm in San Diego and as an assistant controller. His work focuses on business model strategy, business engineering and, more broadly, structural analysis: how companies make money, read from their own filings. He created FourWeekMBA and leads research there and at The Business Engineer, his newsletter on AI and business strategy, with over 95,000 subscribers and more than 1,000 published analyses. His writing has also appeared on Entrepreneur, HackerNoon and Search Engine People. Find him on LinkedIn and Substack. See how we source.
Scroll to Top
Discover more from FourWeekMBA
Subscribe now to keep reading and get access to the full archive.
