OpenAI, Google and Microsoft Back One AI Training Opt-Out: robots.txt Rules and 2027 Deadlines

October 30, 2026. OpenAI, Google and Microsoft announced support for a shared robots.txt signal that lets website owners express preferences about the use of their public content for training generative AI. This is meaningful coordination, but it is not a final internet standard, a universal opt-out already enforced today, or a replacement for existing provider-specific controls.

For business leaders, SEO teams and software publishers, the question is no longer simply whether to block AI crawlers. It is whether to allow training, whether to allow grounding of AI-generated answers, and how those choices affect visibility. These are separate decisions, and different vendors have announced different implementation timelines.

What was announced, and what is not available yet?

Kevin Kelley of Anthropic published an individual Internet-Draft on October 30 proposing the shared __usage_ai-training policy token. The document explicitly has no formal IETF standing; it is a work in progress. OpenAI says it aims to honor the signal in the coming months. Google aims to support it in 2027. Microsoft specifies an effective date of February 1, 2027 as its crawlers encounter the new preferences.

The IETF proposal uses ordinary robots.txt groups and existing parsing rules. It introduces no new directive syntax. Importantly, the token is a preference about content usage, not a crawler-blocking command, security control, copyright license or guarantee that previously collected data will be deleted.

Read the original IETF draft and the OpenAI announcement dated October 30.

The proposed robots.txt syntax

A publisher wishing to express a preference against generative AI training across its site can use this example from the draft:

User-Agent: __usage_ai-training
Disallow: /

Path-specific rules are also possible. A website might prefer that most pages not be used for training while permitting a press-kit directory. Under the draft, a wildcard User-Agent: * group does not automatically express the new AI-training preference. Existing crawler rules remain independent. Providers that do not implement the shared signal may simply ignore the group, so preserve currently effective provider-specific settings.

Do not copy this example without first deciding your policy. For a knowledge publisher or B2B software company, the business value of appearing in AI answers may be significant. The shared signal addresses training preferences; it does not itself provide a universal way to manage AI grounding, referrals or citations.

The exact semantics and limitations are documented in version 00 of the IETF proposal.

Google: a training opt-out may affect Gemini grounding

Google says it plans to interpret the shared training opt-out as if Google-Extended had been disallowed, unless the publisher explicitly specifies Google-Extended. This matters because Google-Extended covers certain Gemini training and grounding uses. Google also says explicitly allowing Google-Extended can keep a website eligible to appear in Gemini Apps responses and receive clicks through grounding. Existing company-specific controls will continue to function.

Google's target is 2027. It would be incorrect to describe the new shared signal as already generally enforced by Google. Equally, do not confuse this with removal from traditional Google Search: review the separate product and crawler rules before changing any website configuration.

Source: Google Search Central announcement, October 30, 2026.

Microsoft: the February 1 and March 1 deadlines matter

Microsoft says its AI Training and AI Grounding preferences begin taking effect from February 1, 2027 as it crawls. The shared training signal can be set in robots.txt for a site or path, and Microsoft adds page-level training overrides. Grounding is separately configurable per page, with Allow, DisallowExtended and Disallow options. The middle option limits grounding to Microsoft-operated experiences rather than third-party grounding services Microsoft supplies.

For publishers relying on NoArchive or NoCache as AI-use controls, Microsoft calls for migration by March 1, 2027. After that date, it says these older settings will no longer be interpreted as training or grounding preferences. The company also describes a temporary Bing Webmaster Tools grounding preference for transition cases; its protection lasts until recrawl or September 30, 2027, whichever comes first.

Microsoft states that the new usage preferences do not change eligibility for traditional Bing search results. However, restricting grounding can reduce visibility in AI-generated experiences, and no choice guarantees traffic, citations or payment. These are announced future-effective changes, not proof that every page has already been reprocessed.

Source: Microsoft Bing official announcement and migration guide, October 30, 2026.

OpenAI: do not discard existing crawler controls

OpenAI says its current robots.txt controls will remain available while the shared signal is implemented. Its crawler documentation distinguishes the purpose of training crawlers from search-related crawling. A publisher seeking discoverability in ChatGPT search while restricting training should understand that difference and preserve relevant controls during the transition. OpenAI has not committed to a specific calendar date beyond the coming months.

Source: OpenAI's official crawler documentation.

Six practical checks before changing a live website

1. Set the business objective. Decide whether to allow model training, AI-answer grounding, both or neither. Do not let a generic 'block AI' setting make this decision by accident.
2. Inventory existing rules. Review robots.txt, Google-Extended, OpenAI crawler directives, Bing page-level metadata, CDN settings and any contractual licensing terms.
3. Map dates to vendors. OpenAI: coming months. Google: 2027. Microsoft: February 1, 2027 implementation and March 1, 2027 legacy-control migration milestone.
4. Validate syntax and access. Test the actual robots.txt file, path matching and page metadata. A usage preference does not replace authentication or stop noncompliant scrapers.
5. Measure visibility. Track search referrals, AI-answer citations and brand visibility. Blocking grounding may reduce exposure even if ordinary search remains available.
6. Assign governance. SEO, legal, security and content owners should jointly approve changes, record the rationale and revisit settings when vendor rollouts are confirmed.

What enterprise teams should take away

AI content governance is becoming an operational policy, not merely a legal paragraph. The strictest rule is not always the most valuable rule: a documentation site, public knowledge base or professional publisher may prefer to restrict model training but retain visibility in AI-generated answers where supported. Other organizations may prioritize content-use restrictions over referrals. Both positions can be rational, provided the trade-offs are explicit.

Bottom line: October 30, 2026 brought coordinated vendor support for a shared AI-training preference, not a completed universal standard. Treat the proposed token as an emerging control, preserve current vendor-specific rules, and plan for Microsoft's 2027 migration dates without claiming that the new signal is universally enforced today.