Backprop-Free Speedrun Cuts a Tiny Model’s Training to 1.6s

Samip Dahal, a cofounder of Q Labs, said on X on 22 October 2026 that in the ten days since the team released the Noise speedrun, “the time to pretrain a transformer went from 1.76 hours to 1.6 seconds, a 4,000x (!!!) speedup.” The training runs never use backpropagation, the standard method for teaching neural networks.

The benchmark is small. The Noise repository fixes a 37.7M-parameter transformer, a FineWeb training set of 10M tokens until 22 October, and one 8xH100 node, and times how fast an entry reaches a validation loss of 4.2. Dahal flagged the limit himself: “Now, scaling is the biggest open question. Some of the tricks that brought the time down to 1.6 seconds will simply not scale.”

Business Pill · BENCHMARKS

A short Business Engineer explainer on benchmarks: why a test score is not your own result. It teaches the general idea only and says nothing about any company in this piece.

What the Noise Repository Measures

Backpropagation, or backprop, trains a model by running it forward to make a prediction and then backward to work out how much each internal number contributed to the error. Noise bans the backward pass. Its rules say: “No backpropagation. No autograd (reverse or forward mode) and no hand-written backward pass.” Every learning signal has to come from running the model forward and measuring the loss.

The methods are what researchers call zeroth-order: instead of computing an exact gradient, they nudge the model many times, measure how much each nudge lowers the loss, and average the nudges weighted by that result. Q Labs’ own method, Dust, “perturbs activations, rewards each perturbation by how much it lowers the loss, and averages the reward-weighted perturbations over a population to estimate the gradient,” its research page says.

• The model. A 37.7M-parameter transformer, the same as in the Dust paper, according to the README.
• The data. 10M tokens of FineWeb until 22 October, now 30M tokens, with at most one pass over the training set.
• The clock. Training time on one 8xH100 node, from the first update to the last, excluding validation.
• The authors. The repository cites Serdar Gülbahar, Bishwas Mandal, Akshay Vegesna and Samip Dahal, and names Gülbahar and Mandal as core contributors.

What Was Posted on X

Samip Dahal, a cofounder of Q Labs, posted on 22 October that in the ten days since the team released the Noise speedrun, the time to pretrain a transformer went from 1.76 hours to 1.6 seconds, which he called a 4,000x speedup, and that Dust was up to 10,000x more compute-efficient than the previous state of the art. He adds that scaling is the biggest open question and that some of the tricks behind the 1.6 seconds will not scale. The 4,000x and 10,000x figures are his; this piece checks the times against the repository’s own records.

The Number: From 1 Hour 46 Minutes to 1.6 Seconds

The repository was created on 12 October 2026. Its first speedrun record, Dust trained with the Adam optimizer, reached the target in 1 hour 46 minutes; record #22, merged on 22 October, did it in 1.6 seconds. By our arithmetic, 1 hour 46 minutes is 6,360 seconds, about 4,000 times the 1.6-second record, in line with Dahal’s figure.

Selected rows from the archived 10M-token speedrun table of the qlabs-eng/noise repository: time on one 8xH100 node to reach validation loss 4.2 on the 10M-token training set, without backpropagation. The repository lists 22 records; the full table names each contributor.

Dahal thanked contributors by name in his post, and the table credits GitHub accounts such as Euraba, erodola, RobertTLange and areu01or00, whose entry set the 1.6-second record.

What the Target Leaves Out

By our reading of the repository’s tables, backprop with Adam reached a validation loss of 3.769 on the same 10M-token setup, below the 4.2 the speedrun asks for. The loss track is closer: its best no-backprop entry reached 3.676, below that backprop figure, in 35 minutes.

From the qlabs-eng/noise README and its 10M-token archive, read on 23 October 2026. Comparisons between rows are our reading of the repository’s own tables.

On 22 October the team moved the benchmark to 30M tokens and a stricter speedrun target of 3.5, and archived the 10M-token records, according to the repository’s commit history. The current 30M-token speedrun record is 1 hour 43 minutes. “To discard those, we’ll be scaling the dataset every time it gets overfit, starting with 30M tokens instead of 10M,” Dahal wrote.

What Q Labs Claims for Dust

Q Labs presents Dust as “the first zeroth-order method that is competitive with backprop at pretraining transformer language models.” The README puts it more cautiously: Dust “rivals backprop under SGD and is 1,000 to 10,000x more efficient than EGGROLL, the state-of-the-art weight-space ES, but still trails backprop under Adam.” The research page says the 1,000 to 10,000 times figure is “based on our extrapolations,” and that in its tests a 243M-parameter model outperformed a 120 times smaller model at most population sizes.

Connecting the Coverage

Our Mirendil piece reported that the lab, founded by former Anthropic researchers, said on 21 October 2026 that its first pretrained model “was built by one researcher in 3.5 months, with less than $10M in compute.” What’s new here: a change to the learning algorithm itself, rather than to the compute budget.

The hardware side is moving too. Our Vera Rubin piece reported that Jensen Huang said at GTC Berlin on 21 October that Vera Rubin is another 30 times beyond Blackwell, with a slide showing 30X in tokens per dollar. Our formal verification piece reported that Theorem’s cost model for AI-written code proofs starts at about $40 per kilobyte of machine code at today’s rates.

All of this sits against the scale of the buildout. Our funding piece, on an episode about an $11 trillion AI buildout, reported SemiAnalysis’s Daniel Nishball arguing for $7 trillion of AI debt outstanding by 2029.

The Structural Read

As we read it, the record is about the learning algorithm, not the chips: every entry ran on the same single 8xH100 node, so the gains came from how the model is updated, according to the repository’s records.

As we read it, the benchmark sets a looser bar than backprop: the repository’s own tables show backprop with Adam reaching a loss of 3.769 on the 10M-token setup, below the 4.2 speedrun target, while its best no-backprop entry on the loss track reached 3.676.

As we read it, the next test is size: the team moved to a 30M-token training set and a 3.5 target on 22 October, where the current speedrun record is 1 hour 43 minutes, according to the README.

Three Implications

FOR AI RESEARCHERS The repository is public and takes submissions by pull request, with the training script as the record, according to its README.

FOR COMPUTE BUYERS Every record so far is for a 37.7M-parameter model on one 8xH100 node; Dahal says some of the tricks behind the 1.6-second record will not scale.

FOR MODEL MAKERS Q Labs says Dust rivals backprop under SGD but still trails backprop under Adam, according to the README.

The Business Engineer Lens

This story maps onto the Business Engineer framework The State of The Inference Economy.

The framework’s premise: “Training was a cost center: episodic, concentrated among roughly twenty frontier model labs, amortizable across trillions of tokens.”

As we read it, Noise works on that cost center: a learning algorithm that needs only forward passes. The records so far are on a 37.7M-parameter model, and the framework, published before Noise, does not discuss it.

What Is Not Established

The records are entries in a public benchmark run by the method’s authors, on a 37.7M-parameter model, 10M training tokens and a loss target that backprop beats; we have not rerun them. Dahal says scaling is the biggest open question and that some of the speed tricks will not scale, and the team has already moved to a larger training set. Q Labs’ efficiency comparison with EGGROLL rests on its own extrapolations. Nothing here shows that frontier-scale training has become cheaper, and nothing here is investment advice.

The Bottom Line

On the Noise benchmark, training a 37.7M-parameter transformer to a fixed loss without backpropagation fell from 1 hour 46 minutes to 1.6 seconds in ten days, about 4,000 times by our arithmetic, matching Dahal’s figure. By our reading of the repository’s tables, backprop reaches a lower loss than that target on the same setup, and the team says scaling is the open question it is testing next.