OpenAI, Google, Microsoft, Anthropic Back One AI Opt-Out

OpenAI said on 30 October that it “will honor a common signal in robots.txt” that lets websites express preferences about “foundational generative AI training”. Google, Microsoft and Anthropic announced support for the same signal the same day, based on an Internet Engineering Task Force (IETF) draft.

The four timelines differ. Microsoft says its new preferences take effect on 1 February 2027, which is 94 days after the announcement by our arithmetic. Google says 2027, OpenAI says “in the coming months” and Anthropic says it “has already started implementing support”.

Short explainer · 4 AI Giants Back One Opt-Out: 94 Days to the First Date

A vertical news Short narrated from this piece’s headline and opening line.

Business Pill · DATA FLYWHEEL

A short Business Engineer explainer on the data flywheel: how usage data can make an AI product better. It teaches the general idea only and does not discuss the robots.txt signal.

What Each Provider Said

OpenAI’s post says: “Previously, website operators would identify individual crawlers and maintain separate rules for each and every provider. Having a shared signal across multiple providers reduces that work for operators.” It adds that the signal “will not replace the controls we already provide through robots.txt”.

Google’s Search Central post, by John Mueller, says that when a site uses the shared token to opt out of training “(and Google-Extended is not explicitly specified), Google plans to interpret that as if the publisher had disallowed Google-Extended”. It says: “We aim to support this new control along with other AI providers in 2027”.

Microsoft’s Bing post says it is “building on the shared AI Training preference and adding a separate AI Grounding preference”. It says the new preferences “will be effective on February 1, 2027, as Microsoft crawls”, and that both training and grounding “default to Allow when no applicable preference is published”.

The IETF draft is written by K. Kelley, listed at Anthropic. In a message to the IETF ai-control list, Kelley wrote: “I’m also excited to share that Anthropic has already started implementing support for this draft.”

How the Signal Works

The draft uses ordinary robots.txt syntax. A site writes a rule group whose User-Agent line names a “policy token”. The draft’s own example is:

User-Agent: __usage_ai-training
Disallow: /
Allow: /press-kit/

The draft says this “expresses a preference against AI training for all paths except those under /press-kit/”. Kelley’s message adds that wildcard rules never match the token, so “a robots.txt file that doesn’t use this mechanism, as is true of every one published today, expresses no preference at all”.

The draft is an Internet-Draft dated 30 October 2026 that expires on 3 May 2027. Google calls it a “draft proposal”, and Microsoft says the working group’s efforts “continue toward a final standard”.

What It Means for Publishers

Microsoft’s post states the trade-off plainly: “Allowing training can help improve future generative models. Neither choice guarantees inclusion, traffic, or payment.” It adds that if a site selects “DisallowExtended” or “Disallow” for grounding, it “will see reduced brand visibility in AI experiences”.

Google says sites that “specifically allow Google-Extended are eligible to appear in responses and receive clicks via grounding in Gemini Apps”. It also says the proposal “is consistent with the EU Code of Practice for General Purpose AI (GPAI) Model Providers”.

We have covered publishers’ terms with AI companies before, including the UK competition probe into Meta’s terms on publisher content and Cloudflare’s split of crawlers by purpose in September.

The Structural Read

As we read the four posts, the change moves the opt-out from a list of company-specific rules to one shared line, while OpenAI, Google and Anthropic say their own controls, such as Google-Extended, keep working alongside it, and Microsoft says its new preferences will replace NoArchive and NoCache for training and grounding.

As we read Microsoft’s post, training and grounding are now separate choices: a site can refuse training and still allow its pages to inform AI answers, and Microsoft says choosing Disallow or DisallowExtended for grounding reduces brand visibility in AI experiences.

As we read the dates, the shared signal arrives on different clocks: Anthropic says it has started implementing support, Microsoft gives 1 February 2027, Google gives 2027 and OpenAI gives no month.

Three Implications

FOR PUBLISHERS The draft’s example is a three-line robots.txt group; Kelley says a file that does not use the token expresses no preference at all.

FOR MICROSOFT’S SITES Microsoft says sites using NoArchive or NoCache should set the new preferences by 1 March 2027 to keep their restrictions.

FOR LICENSING TALKS Microsoft says neither choice guarantees inclusion, traffic or payment, so the signal records a preference rather than a price.

The Business Engineer Lens

This story maps onto the Business Engineer framework The Four Intelligence Moats.

The framework puts it this way: “The Corpus Moat — built by pretraining, eroding as the public web is exhausted and synthetic data closes gaps.”

As we read it, a shared training opt-out is a control on that corpus: it lets each site state whether its pages may be used for training. The framework was published in July 2026 and does not discuss the IETF signal.

What Is Not Established

None of the four posts says how many sites will use the token. OpenAI and Google give no month for support. The draft is not an IETF standard; as an Internet-Draft it may be updated, replaced or obsoleted.

The Bottom Line

OpenAI, Google, Microsoft and Anthropic say they will honour one robots.txt line, __usage_ai-training, as a training opt-out across providers. Microsoft names 1 February 2027, 94 days away by our arithmetic. The others give 2027, “the coming months” or work already under way.