logo
About Contact

Google and Microsoft Commit to New AI-Driven Website Crawling Standards with Opt-Out Options

AI News

Google and Microsoft Commit to New AI-Driven Website Crawling Standards with Opt-Out Options

Google, Microsoft join AI training opt-out plan for websites

Dataconomy

Google, Microsoft join AI training opt-out plan for websites

The voluntary robots.txt token governs content use, leaves crawl access unchanged, and cannot undo past data collection.

Google and Microsoft among those signing up to OpenAI's plan for a new website crawling standard — including opting-out of internet scraping

TechRadar

Google and Microsoft among those signing up to OpenAI's plan for a new website crawling standard — including opting-out of internet scraping

One robots.txt line could soon tell Google, Microsoft, OpenAI, and Anthropic not to train on your site, but it won't stop their bots from crawling it.

Coverage dates

First reported:

First saved on ClustZ:

Coverage updated:

We use cookies to improve your experience and analyze site traffic. By clicking "Accept", you agree to our use of cookies. See our Privacy Policy.