DoorDash’s Nvidia Test: RL Scored 3x Distillation on Puzzles

DoorDash’s AI research team post-trained Nvidia’s Nemotron Nano 9B on three new skills in a row and found that GRPO, a reinforcement learning method, scored 95.9% on a logic-puzzle task against 31.6% for self-distillation, about 3 times as much, by our arithmetic. The results are in a DoorDash engineering blog post published on 2 November.

The post also measured what the model forgot. After the three stages, the self-distilled model was 30 points below the base model on logic, 21 on instruction following and 15 on math, the post says. GRPO ended 4, 3 and 7 points up. By our arithmetic, that puts the two methods 34, 24 and 22 points apart.

What DoorDash Tested

The team trained the model in three stages: tool use, then science, then Knights & Knaves logic puzzles, each starting from the previous stage’s weights. It scored every stage across twenty benchmarks in all: held-out tests of the three trained tasks, plus five themes (instruction following, code, math, logic and knowledge).

Both methods update all model weights and train for the same number of iterations, the post says. The main comparison runs with the model’s thinking mode off; a separate self-distillation chain runs with thinking on.

The reason is operational. “At DoorDash, new tools, domains, and tasks keep expanding the surface area we need our agents to cover,” the authors, Chaitanya Ahuja and Sebastian Zuba of DoorDash AI Research, write. “Each round of post-training has to add those capabilities without degrading the ones already in production.”

The post also states the cost of RL. RL needs a usable reward signal and several sampled answers per prompt, “requiring more than 5x the compute and training time relative to SFT”, the post says, referring to supervised fine-tuning.

What Each Method Learned and Lost

Both methods learned the first two skills. Tool use rose 8 points under self-distillation and 20 under GRPO; science rose 33 and 29. On Knights & Knaves, self-distillation ended at 31.6%, below the base model’s 47.7%, the post says.

Self-distillation’s losses grew stage by stage: instruction following fell 15, then 20, then 21 points below the base model, and logic 12, then 25, then 30. GRPO “never drops more than 0.2 points on any theme at any stage”, according to the post.

Knowledge barely moved. Neither method shifted the knowledge theme by more than 4 points; the largest single drop was MMLU-Pro under self-distillation, from 62.9% to 54.0%.

Part of the loss is formatting. Ignoring answer format cuts self-distillation’s BBH logic loss from 57 points to 37, the post says, so it still ends 37 points below the base model there, while GRPO ends 5 points above.

How the Answers Changed

In the thread on X, DoorDash wrote that GRPO “learns to verify its own work”. On AIME 2025, the share of answers with an explicit self-check rose from 1% to 22% under GRPO and to 4% under self-distillation.

Self-distillation cut the median AIME 2025 answer from 856 tokens to 597; GRPO kept it at 860. With thinking on, the self-distilled median fell from 10,092 tokens to 4,768, and AIME 2025 accuracy fell from 59.2% to 26.2% with format ignored.

The traces show why, according to the authors. On one AMC 2023 problem, the self-distilled model cited an example only its teacher had seen: “Since the example arrives at 270, I will follow its logic to ensure consistency”. The base model and GRPO both reached 3159.

Training the Skills Together

Shuffling the three datasets into one self-distillation run softened most of the forgetting: logic dropped 4 points instead of 30, instruction following 6 instead of 21, math 8 instead of 15 and code 2 instead of 5.

It did not bring back the long answers. AIME 2025 answers stayed short, at a median of 495 tokens, and only 2% checked their work, the post says. The authors did not test methods that put self-distillation inside an RL loop.

Business Pill · Fine-Tuning

An evergreen Business Engineer explainer on fine-tuning: how a general AI model is adapted to one job. It teaches the general idea only and does not discuss DoorDash, GRPO or self-distillation.

Connecting the Coverage

Enterprises tuning Nvidia’s open models also appeared in Palantir: Tuned Nvidia Model Tops GPT-6 Astra at 1/200 Price and Palantir Pitches a Model Factory Where Clients Own Weights. A lab approach to learning without forgetting was in Google Gives LLMs a Sleep Phase to Learn Past Their Cutoff.

DoorDash’s consumer-facing view of agents was covered in Amazon, DoorDash CEOs Say Few Shoppers Buy Through AI Agents.

The Structural Read

As we read the post, the forgetting compounds: each new self-distillation stage left instruction following and logic further below the base model, so a company that adds skills one release at a time would see the losses stack, at least in this setup.

As we read the traces DoorDash published, the failure is procedural rather than factual: knowledge moved no more than 4 points, while the self-distilled model copied a teacher’s example it had never seen and the last dataset’s answer format.

As we read the mixing result, the order of training data matters on its own: shuffling the three datasets cut self-distillation’s logic loss from 30 points to 4, though it still lost 6 on instruction following and 8 on math, the answers stayed short and only 2% checked their work.

Three Implications

FOR ENTERPRISES TUNING OPEN MODELS DoorDash’s test scored every stage on five themes of prior capabilities, not only on the new skill; self-distillation’s losses there reached 30 points on logic.

FOR NVIDIA The test ran on Nvidia’s Nemotron Nano 9B, which DoorDash post-trained on all model weights for three new skills, according to the post.

FOR TRAINING BUDGETS The post says RL requires a usable reward signal and multiple sampled responses per prompt, and more than 5 times the compute and training time of supervised fine-tuning.

The Business Engineer Lens

This story maps onto the Business Engineer framework The Four Intelligence Moats.

The framework lists four moats, one of them this: “The Verifier Moat — built by RLVR on domain-specific reward signals, building into a new asset class for reasoning-heavy verticals.”

As we read it, DoorDash’s test sits on that layer: GRPO needs a reward signal that can grade an answer, and in this experiment it was the method that added skills without losing old ones. The framework does not discuss DoorDash, Nvidia or self-distillation.

What Is Not Established

This is one experiment on one 9-billion-parameter model, three training sets and one task order, run by DoorDash’s own researchers; it has not been independently replicated. The post does not say which DoorDash systems, if any, use GRPO in production, give a dollar cost for either method, or report GRPO’s exact scores on code and knowledge. Some self-distillation answers were graded leniently for format. We did not contact DoorDash or Nvidia.

The Bottom Line

In DoorDash’s test on Nvidia’s Nemotron Nano 9B, GRPO learned the new skills. It never fell more than 0.2 points below the base model on any theme at any stage and ended up on math, logic and instruction following. Self-distillation lost 30 points on logic and 21 on instruction following. The post puts RL’s compute and training time at more than 5 times supervised fine-tuning’s.