OpenAI’s open-weight model gpt-oss-20b found about 2.6 times as many held-out materials when every training example was a fact seen only once, compared with none, in a paper by researchers at Johns Hopkins University, Microsoft and Apple posted on arXiv on 2 November. Its discovery rate rose from about 1.8% to about 4.8%, by our reading of the paper’s chart.
Google’s open-weight gemma-3-12b-it rose about 3.2 times, from about 1.3% to about 4.2%. The paper calls these once-seen facts monofacts, “the facts observed once during training”, and names their share of the training data “a key factor governing discoveries in language models”.
Business Pill · Verification Cost
A short explainer of verification cost: why AI output that is cheap to make can still be costly to check. It teaches the general idea only and does not discuss this story.
The key insight: As we read it, the paper puts a cost on a routine data choice: in its materials test, a training set made entirely of facts seen once lifted the discovery rate about 2.6 times for OpenAI’s gpt-oss-20b and about 3.2 times for Google’s gemma-3-12b-it, by our arithmetic on the paper’s chart, while repeating the data erased the gain.
The authors trained models in two fields, materials science and regulatory genomics. In materials, a model had to generate a chemical formula that met a set of target properties. Discovery meant exactly recovering a target the model had never seen: the paper trains on a 14 November 2024 snapshot of the Materials Project database and tests on formulas that first appear in its 25 September 2025 snapshot, after both models’ knowledge cutoffs.
They varied the monofact rate of the training set from 0% to 100% in steps of 10 points. In materials, training ran in two stages, the paper says: first on datasets of 10,000 examples, then on 4,000 examples with reasoning traces distilled from Claude Opus 5. The authors say that second stage more closely reflects how researchers use language models for discovery.
New Johns Hopkins paper finds that LLMs make more scientific discoveries when more of their training facts appear only once.
discovery rates rose steadily as the share of once-seen facts in training went from 0% to 100%, for 2 model families.
Discovery and hallucination both mean producing something the model never saw in training. Earlier theory tied hallucination to “monofacts”, facts seen exactly once, and treated that unseen space as mostly false.
The authors extend that theory to show monofacts also set how much a model can discover. They fine-tuned gpt-oss-20b and gemma-3-12b-it on materials and DNA datasets, varying the monofact share from 0% to 100%.
Discovery, scored as exactly recovering materials or DNA sequences held out from training, rose steadily with the monofact share in every setting. Training for 3 or 5 epochs, which turns single sightings into repeats, made the trend disappear.
When building science models, keep rare one-off examples and avoid repeating data, and expect more hallucinations to come with more discoveries.
Rohan Paul summarised the Johns Hopkins paper on X on 5 November, with an image of its title page: discovery rates rose as the share of once-seen facts in training went from 0% to 100% for two model families, and training for 3 or 5 epochs made the trend disappear. His closing advice to keep rare one-off examples is his own reading; the figures in this piece come from the paper.
What They Found
The paper says discovery increased monotonically with the monofact rate in both fields, for both model families, after both training stages, and whether the models had one attempt or 16. The 2.6x and 3.2x figures are the second-stage results for materials with one attempt, our arithmetic on values read from the paper’s Figure 2.
The same lever raises errors. The authors expected that “higher monofact rate increases both discovery and hallucination” when training and test data come from the same distribution, and the paper reports that both rose on held-out examples from the training snapshot.
Repetition removed the effect. When the researchers trained gpt-oss-20b for three or five passes over the same data instead of one, the paper says the rising trend disappeared, and at high monofact rates one pass found more than repeated passes. The authors say this rules out training-set diversity alone as the explanation.
Why It Matters for AI Science Companies
The finding speaks to a choice every lab building science models makes: how much to deduplicate data and how many times to train on it. The paper’s conclusion says “monofact rate signals that the training data has not exhausted the underlying distribution, leaving room for unseen outcomes to be discovered”.
As we read the results, deduplication and multiple training passes are not neutral for a science model: the paper reports that seeing each fact more than once removed the rising trend.
As we read the hallucination finding, the same setting that raises discoveries raises wrong answers, so a lab that keeps rare facts would also need a way to check outputs, such as the exact-match tests the paper uses.
As we read the setup, the second training stage used reasoning traces distilled from Claude Opus 5, so frontier-model outputs were part of how the smaller open models were taught.
Uzunoglu et al., arXiv 2611.03617, 2 November 2026
“Making such discoveries, however, requires LLMs to venture beyond what they have already observed”
Three Implications
FOR AI SCIENCE COMPANIES The paper reports more discoveries when more training facts appear only once, in both materials science and regulatory genomics.
FOR DATA TEAMS The paper reports that training for three or five passes instead of one erased the trend in its materials test.
FOR READING THE NUMBERS This is a preprint covering two open-weight models and two tasks; the 2.6x and 3.2x figures are our arithmetic on values read from the paper’s chart.
The framework says: “The transformer is a commodity now; what changes between paradigms is where intelligence accumulates and who can capture it.”
As we read it, the Johns Hopkins paper locates part of that accumulation in a training-data statistic: how many facts a model sees exactly once. The framework does not discuss this paper.
What Is Not Established
This is a preprint on arXiv covering two open-weight models, gpt-oss-20b and gemma-3-12b-it, fine-tuned on two scientific tasks. Discovery is measured as exactly recovering targets held out from training, not as new experiments, and in materials with one attempt, absolute rates stay under 5%. The authors say their theoretical bounds depend on parameters that are difficult to estimate in practice. The 2.6x and 3.2x figures are read from a chart and are approximate.
This Week In The AI Supercycle
Anthropic’s Reality Check, The Capital Squeeze, & The New World Order
This week’s issue: for two years the AI build-out was priced as a growthstory. This week it started being priced as a credit story.
In the Johns Hopkins paper, training on facts seen only once lifted materials discovery about 2.6 times for OpenAI’s gpt-oss-20b and about 3.2 times for Google’s gemma-3-12b-it, by our arithmetic on the paper’s chart. The paper says hallucination rose with it on data from the training distribution, and that repeated training erased the effect.
A note on sourcing. Figures and quotations are from the paper by Uzunoglu and colleagues posted on arXiv on 2 November 2026 (2611.03617). Discovery rates were read from the paper’s Figure 2(a) plot and are approximate; the 2.6x and 3.2x ratios are our arithmetic. The study was flagged on X by Rohan Paul.
Gennaro Cuofano is a CRO and tech executive who has worked in AI since late 2015, starting with bringing NLP, natural-language generation, voice and chatbot products to the marketing industry. His background is in law and finance: he holds a Master's degree in Law and an International MBA with an emphasis on corporate finance (LUISS Business School and the University of San Diego, 2012), and worked as a financial analyst at a real-estate investment firm in San Diego and as an assistant controller. His work focuses on business model strategy, business engineering and, more broadly, structural analysis: how companies make money, read from their own filings. He created FourWeekMBA and leads research there and at The Business Engineer, his newsletter on AI and business strategy, with over 95,000 subscribers and more than 1,000 published analyses. His writing has also appeared on Entrepreneur (for example, How to Digitize a Family Business in Europe) and HackerNoon. Find him on LinkedIn and Substack. See how we source.
