OpenAI: Next 0.08 Points Bring Unknown Skills, Weaker Audits

OpenAI’s chief scientist Jakub Pachocki says the next stage of model progress will bring capabilities nobody can name yet. “I don’t know what those capabilities will be, and we may not be testing for them today,” he wrote on Sunday in “Perplexity and Intelligence”.

That stage is a log-perplexity window of about 0.68 to 0.70, 0.06 to 0.08 points below the model that began new maths results. By our arithmetic, that gap is 40% to 53% of the previous 0.15-point step.

He also expects “further degradation in currently dominant forms of alignment auditing.” Our main story on the post is here; we track scores on our AI model benchmarks page.

Business Pill · The Model and the Harness

An evergreen Business Engineer explainer. It teaches the general idea only and does not discuss this case.

The key insight: OpenAI’s chief scientist expects unknown new capabilities in a 0.68 to 0.70 log-perplexity window, 0.06 to 0.08 points below its 0.76 maths model (40% to 53% of the previous step, by our arithmetic), along with weaker alignment auditing and more evaluation awareness.

What Came at Each Level

Pachocki ties past capabilities to OpenAI’s loss scores. GPT-3.5, at about 0.99, “was capable of fluent conversation and formed the basis for the original ChatGPT.” GPT-4, at about 0.91, “was proficient at utilizing chain-of-thought reasoning productively.”

Below 0.8, he wrote, “we have seen the emergence of transformative capabilities in fields like cybersecurity and mathematics.” Our explainer on what the score measures is here.

What OpenAI’s chief scientist says came at each approximate loss level, and what he expects in the 0.68
What OpenAI’s chief scientist says came at each approximate loss level, and what he expects in the 0.68 to 0.70 window. The 40% to 53% share is our arithmetic.

The Next Window

“Extrapolating from recent progress, we should expect further transformative capabilities as we enter the ~0.68-0.7 log-perplexity window,” Pachocki wrote.

By our arithmetic, perplexity in that window would be about 1.97 to 2.01, against about 2.14 for the maths model. The post does not give a date for reaching it.

The Warning on Audits

In the same passage he wrote: “we should also expect further degradation in currently dominant forms of alignment auditing and further increases in evaluation and training awareness.”

Evaluation awareness means a model noticing it is being tested. As we read it, that makes test results harder to trust, which is why Pachocki ties the warning to new shared rules.

He also wrote that AI will help alignment and monitoring, but “that progress is unlikely to be fast enough to allow the full potential acceleration in pretraining.”

What He Wants Before Then

Pachocki wants developers to “jointly endeavour to establish shared norms for how to evaluate such models and red-team safety cases for them by the time they are first used, including shared transparency standards.”

He then proposes a first limit on a metric tracked across labs, held until monitoring reaches a milestone such as “a successor to CoT monitors.” How rivals’ own rules compare is here.

Flow diagram of Pachocki's proposal: shared evaluation norms, red-teamed safety cases, a first limit on a metric tracked across labs, then a monitoring milestone before successive limits. Source: OpenAI
Pachocki’s proposal in four steps, from OpenAI’s post: shared evaluation norms, red-teamed safety cases, a first cross-lab limit, then a monitoring milestone before the next limit.

The Structural Read

As we read it, the warning on auditing is the reason the proposal is urgent: if tests get less reliable as models improve, limits have to be agreed before, not after.

As we read it, a gap of roughly half the previous step suggests OpenAI sees the window as near, though the post gives no date.

Jakub Pachocki, OpenAI Chief Scientist, 22 Nov 2026

“I don’t know what those capabilities will be, and we may not be testing for them today.”

Three Implications

FOR AI LABS Pachocki asks for shared evaluation norms and red-teamed safety cases by the time such models are first used.

FOR SAFETY TEAMS He expects further degradation in currently dominant forms of alignment auditing, per OpenAI’s post.

FOR REGULATORS The post proposes a first limit measured by a metric tracked across labs, with a monitoring milestone before going beyond it.

What Is Not Established

The values are OpenAI’s approximations on an internal dataset that no one else can check. The post names no date for the window and no number for a first limit.

The link between loss scores and capabilities is Pachocki’s reading of OpenAI’s own history. His maths results and 2027 forecast are covered here.

This Week In The AI Supercycle

A World Without an Order, and an AI Central Bank Without a Successor

This week’s issue: the week AI memory stayed largely sold out while the people holding the shares paid: why the AI Supercycle is not a dot-com replay, read through geopolitics, macro and technology, with the gap between building and financing the widest we’ve recorded.

Read the full issue →

Business Engineer Framework

The AI Supercycle

The Business Engineer map of who builds, finances and captures value in AI.

Read the Map of AI →

The Bottom Line

OpenAI says the next 0.06 to 0.08 points will bring abilities it may not be testing for, while its main audit tools get weaker. That is the case it makes for agreeing limits now, before the window arrives.

97,000+ executives read Business Engineer for the AI strategy frameworks cited by ChatGPT, Claude, and Perplexity.

A note on sourcing. All quotes and log-perplexity values are from OpenAI’s post “Perplexity and Intelligence” by Jakub Pachocki, 22 November 2026, which gives approximate values on an internal held-out dataset. The gaps, the 40% to 53% share and the conversion to perplexity are our arithmetic.

Sources: OpenAI (Jakub Pachocki, Chief Scientist), ‘Perplexity and Intelligence’, 22 Nov 2026

Scroll to Top
FourWeekMBA