Researchers at FAIR, Meta Superintelligence Labs, posted a paper on arXiv on 27 October 2026 describing a test that a model provider can run through an API alone to show that a rival model was distilled from its watermarked outputs. At a 0.1% false-positive rate, the paper says, the method detects 79% of distilled models, against 29% for the state-of-the-art black-box detector.
By our arithmetic, that is 2.7 times the detection rate of the earlier test. The paper, picked up on X by DAIR.AI on 29 October, comes as distillation is in the news: The Information reported that DeepSeek and Moonshot AI relied heavily on outputs of OpenAI’s GPT-6 Astra, as we reported in OpenAI Rivals DeepSeek, Kimi Distilled GPT-6 Astra.
Short explainer · Distillation detection
A vertical news Short narrated from the published headline and opening lines of this piece.
Business Pill · The Model and the Harness
An evergreen Business Engineer explainer on models and the systems around them. It teaches the general idea only and does not discuss this paper.
The key insight: By our arithmetic from the paper’s own figures, Meta FAIR’s API-only test flags 2.7 times as many distilled models as the earlier token-level test at a 0.1% false-positive rate, but only on the domain the copy was trained on, and only if the original provider watermarked its outputs first.
The paper, “Watermark Basins: Black-Box Detection of Distilled LLMs”, starts from a provider that watermarks its teacher model’s outputs. Its abstract says that during fine-tuning, “watermarked teacher traces embed a distinct, key-specific geometric signal in parameter space that lies orthogonal to the ordinary task-specific fine-tuning subspace”.
The detector, WM-Basin, “queries the suspect model through its API, without access to its weights or logits”. It turns the suspect’s text into gradients on a fixed clean probe model, removes the ordinary fine-tuning directions, and tests whether what is left lines up with the watermark key, the abstract says.
Share of distilled student models detected at a 0.1% false-positive rate: the token-level black-box baseline vs WM-Basin, 29% to 79% at n-gram=1 and 1.3% to 45% at n-gram=3, as reported in the paper. The 2.7x and ~35x multiples are our arithmetic. Source: arXiv 2610.37266, FAIR, Meta Superintelligence Labs, 27 Oct 2026.
What Was Posted on X
Banger paper from Meta FAIR on catching distilled models.
(bookmark it)
At a 0.1% false-positive rate, their detector flags 79% of models distilled from watermarked outputs. The best token-level test flags 29%.
When a student trains on watermarked teacher traces, the watermark key leaves a specific direction in the student’s weights. That direction sits apart from the normal fine-tuning directions, so it stays even when the student’s text shows no token-level trace of the watermark.
WM-Basin only needs API access. It turns the suspect model’s outputs into gradients on a clean probe model, removes the task directions, and checks what remains against the watermark key.
The signal holds after quantization, pruning, and RL post-training. Students range from 135M to 8.2B parameters.
DAIR.AI on X, 29 Oct: at a 0.1% false-positive rate, the detector flags 79% of models distilled from watermarked outputs, against 29% for the best token-level test; it needs only API access, and the signal holds after quantization, pruning and RL post-training.
The Numbers
The paper reports results among 1,200 student models distilled from DeepSeek-R1-Distill-Qwen-14B on OpenR1-Math reasoning traces. At a 0.1% false-positive rate, WM-Basin detects 79% of distilled models at n-gram=1 and 45% at n-gram=3, against 29% and 1.3% for the state-of-the-art black-box detector. Students range from 135 million to 8.2 billion parameters.
By our arithmetic, 45% against 1.3% is about 35 times the baseline at n-gram=3. The paper says “Quantization, pruning, and reinforcement-learning post-training all leave detection intact”.
Its teacher table also covers OpenAI’s open-weight gpt-oss-120B as a teacher: at n-gram=3 and a 0.1% false-positive rate, WM-Basin flags 30% of students against 2% for the baseline, about 15 times, by our arithmetic. For a Llama-3.1-8B teacher the figures are 100% and 64%.
Share of 100 students distilled on math that WM-Basin flags at a 0.1% false-positive rate (n-gram=1), by the domain audited: 96% on OpenR1-Math, 56% on MATH, 3% on GSM8K, 0% on code and chat. Source: arXiv 2610.37266, Table 5, 27 Oct 2026.
Where It Stops Working
The test depends on the domain the verifier audits. On the same 100 suspects, all distilled on math, WM-Basin flags 96% at a 0.1% false-positive rate (n-gram=1) when audited on OpenR1-Math, 3% on GSM8K and 0% on code and chat, according to the paper’s Table 5. The authors write that “a low score elsewhere means nothing was found there, not that the model is clean”.
They also say “Continued training on clean data can reshape that distribution and therefore degrade black-box detection”. The ethics statement says “every experiment runs on models we host ourselves”, and that the teachers, students and data are publicly released artefacts.
The Structural Read
As we read the paper, the evidence a provider can gather depends on a choice it makes before any copying happens: whether to watermark its model’s outputs.
As we read Table 5, a clean result on code or chat says nothing about a model distilled on math; the authors write that a low score elsewhere means nothing was found there.
By our arithmetic, the gap is widest where the old test was weakest: at n-gram=3, 45% against 1.3%, about 35 times.
FAIR, Meta Superintelligence Labs, arXiv 2610.37266
“a low score elsewhere means nothing was found there, not that the model is clean”
Three Implications
FOR LABS SELLING API ACCESS The paper’s method needs the provider’s watermark key and teacher traces generated under it; it does not work on outputs that were never watermarked.
FOR LABS ACCUSED OF DISTILLING The authors say quantization, pruning and reinforcement-learning post-training all leave detection intact in their tests, which used students up to 8.2 billion parameters.
FOR AUDITORS The paper says a hit tells the verifier on which domain the suspect’s text carries the watermark, so an audit has to match the domain the copy was trained on.
The framework puts it this way: “The transformer is a commodity now; what changes between paradigms is where intelligence accumulates and who can capture it”.
As we read it, distillation is one way a rival captures intelligence another lab accumulated, and the paper describes a way for that lab to show it in a model’s weights. The framework does not discuss this paper or distillation detection.
The test only works if the provider watermarked its outputs before the distillation happened. The paper’s experiments use math reasoning and students up to 8.2 billion parameters, and run on models the authors host. We read the paper’s abstract, introduction, results tables, conclusion and ethics statement. The Information’s report on DeepSeek and Moonshot is paywalled and we read only its headline; the paper does not test those labs’ models. Ratios marked as ours are our arithmetic.
This Week In The AI Supercycle
Execution Fault Lines, Concentrated Demand, and Geopolitical Firewalls
This week’s issue: the AI build is not short of demand for its parts. It is short of cheap money, and its end revenue rests on a narrow base of large, often venture-funded buyers.
A FAIR team at Meta Superintelligence Labs reports an API-only test that flags 79% of models distilled from watermarked outputs at a 0.1% false-positive rate, 2.7 times the earlier test by our arithmetic. The paper’s own limits are that the teacher must be watermarked and the audit must match the domain the student was trained on.
A note on sourcing. Facts come from the arXiv paper Watermark Basins: Black-Box Detection of Distilled LLMs (2610.37266, submitted 27 October 2026, affiliation FAIR, Meta Superintelligence Labs): we read its abstract, introduction, results tables, conclusion and ethics statement on 29 October. The DAIR.AI post on X surfaced it. We did not contact Meta or the authors. Detection rates are the paper’s. Multiples marked as ours are our arithmetic.
Gennaro Cuofano is a CRO and tech executive who has worked in AI since late 2015, starting with bringing NLP, natural-language generation, voice and chatbot products to the marketing industry. His background is in law and finance: he holds a Master's degree in Law and an International MBA with an emphasis on corporate finance (LUISS Business School and the University of San Diego, 2012), and worked as a financial analyst at a real-estate investment firm in San Diego and as an assistant controller. His work focuses on business model strategy, business engineering and, more broadly, structural analysis: how companies make money, read from their own filings. He created FourWeekMBA and leads research there and at The Business Engineer, his newsletter on AI and business strategy, with over 95,000 subscribers and more than 1,000 published analyses. His writing has also appeared on Entrepreneur (for example, How to Digitize a Family Business in Europe) and HackerNoon. Find him on LinkedIn and Substack. See how we source.
