Microsoft NextLat Revision Adds Mamba; Drafts 3.3x Faster

A fifth version of the Microsoft Research paper on Next-Latent Prediction, or NextLat, posted to arXiv on 11 November, adds results on Mamba, a linear-recurrence model, and a section on scaling the method to large pretraining runs. In the paper, 1.3B-parameter models trained with NextLat draft their own next tokens up to 3.3x faster than standard decoding.

By our arithmetic from the paper’s Table 4, NextLat accepts 4.05 draft tokens per step on average across four domains, 2.2x the 1.86 of the closest rival method. First author Jayden Teoh, an MIT CSAIL PhD student, wrote on X on 13 November that the scaling section, on multi-token belief states, came out of efforts to bring NextLat to the next generation of Microsoft’s Phi models.

Business Pill · Tokens Explained

An evergreen Business Engineer explainer on tokens, the unit a model generates and an inference bill counts. It teaches the general idea only and does not discuss NextLat.

What NextLat Changes in Training

NextLat adds a second objective to next-token training: the transformer also learns to predict its own next hidden state, given the next token. The paper says this pushes the model toward belief states, compressed summaries of the past that are enough to predict the future, while leaving the architecture, parallel training and inference unchanged.

The paper lists ten authors, from Teoh to John Langford, under Microsoft Research. Teoh’s post called the method “NextLat has been my baby that I’ve worked on for the past year and a half”; the paper’s first version went up in November 2025.

What Was Posted on X

Jayden Teoh, the paper’s first author and an MIT CSAIL PhD student, announced the fifth version on 13 November and listed what it adds. We quote his post for his own account; the results come from the paper.

What the Fifth Version Adds

A new section on scaling proposes multi-token belief states: instead of one hidden state, the belief state becomes a window of the last few hidden states, and the paper suggests parameterising the predictor with DeepSeek-V3’s multi-token prediction module, using sliding-window attention. The authors write that preliminary experiments with a separate change, normalising the token embedding input, suggest it substantially improves perplexity.

The revision also trains Mamba with NextLat on a graph path-finding task and reports that it significantly improves Mamba’s solve rate; the paper gives that result as a figure. Teoh’s post lists more ablations and references the team had missed, and the paper now carries the NeurIPS 2026 header.

Faster Drafts, Modest Accuracy

On 1.3B-parameter models trained on 100B FineWeb-Edu tokens, NextLat with a two-step horizon reaches a 3.32x speedup on books text, against 1.90x for the best rival in Table 4. Accuracy moved less. The paper reports NextLat raises average zero-shot accuracy to 59.21 from 58.82 for standard training, and says “these improvements are modest and not consistent across tasks”.

Teoh linked the Phi effort to a blog post by Langford: “These came out of our efforts to bring NextLat to the next generation of Phi models, though I’m sure y’all can guess how that went from @JohnCLangford’s recent blog post.” In that post, dated 6 November, Langford wrote that Microsoft’s AI Frontiers group was reorganised into Microsoft AI in early September, that the core philosophy there is “scaling, and aggressively not research”, and that he left.

He named “the Next Latent objective which enables extrapolation” among the group’s work. We covered Langford’s exit from Microsoft and three Big Tech AI research groups reorganised since 2023.

The Structural Read

As we read it, the paper’s clearest gain is at inference, not in accuracy: average zero-shot accuracy moves from 58.82 to 59.21, while the speedup on books text reaches 3.32x. Langford’s post describes the group that built it as reorganised into a unit whose core philosophy, in his words, is scaling rather than research.

Three Implications

FOR INFERENCE COST The paper reports self-speculative decoding speedups of up to 3.3x over standard decoding on 1.3B models.

FOR MODEL ARCHITECTURE The paper says NextLat requires no architectural changes and also improved Mamba on a graph path-finding task.

FOR MICROSOFT RESEARCH Langford wrote on 6 November that the AI Frontiers group was reorganised into Microsoft AI in early September and that he left.

The Business Engineer Lens

This story maps onto the Business Engineer framework The State of The Inference Economy.

The framework says: “Inference is a revenue engine: continuous, distributed across millions of applications and agents, incurring marginal cost with every query.”

As we read it, a training objective that lets a model draft its own tokens up to 3.3x faster acts on that marginal cost without new hardware. The framework predates the paper and does not discuss NextLat.

What Is Not Established

Neither the paper nor Teoh’s post says whether NextLat was used in any Phi model, and Microsoft has not commented in what we read. The scaling section’s multi-token belief states are proposals with preliminary experiments, not results at scale. The largest model in the paper has 1.3B parameters; the authors write that larger sizes might be needed to see bigger accuracy gains. We did not run the code.

The Bottom Line

The fifth NextLat paper reports drafts up to 3.3x faster on 1.3B models and adds a Mamba test and a plan for multi-token belief states written while the team tried to bring the method to Phi. By our arithmetic it accepts 2.2x as many draft tokens per step as the nearest rival. The open question is whether anyone trains it at frontier scale.