About Contact

AI Titans Unite to Provide Publishers with New Scraping Opt-Out Options

AI Titans Unite to Provide Publishers with New Scraping Opt-Out Options

AI giants promise publishers new way to block some scraping
AI giants promise publishers new way to block some scraping

OpenAI, Google, Microsoft and Anthropic pledged on Friday to give media outlets and other websites a single way to prevent their content from being scraped to train artificial intelligence (AI)...

AI giants promise publishers new way to block some scraping
AI giants promise publishers new way to block some scraping

Publishers currently have to implement multiple blocking commands for every bot that crawls the web to collect data

Google joins OpenAI and Anthropic in adopting new AI scraping controls
Google joins OpenAI and Anthropic in adopting new AI scraping controls

Google, OpenAI, Anthropic and Microsoft back a shared robots.txt standard giving publishers greater control over AI training crawlers.

Google, Microsoft, OpenAI and Anthropic back one AI training opt-out
Google, Microsoft, OpenAI and Anthropic back one AI training opt-out

Google, Microsoft, OpenAI and Anthropic will honour one shared AI training opt-out in robots.txt. It lets a website keep its content out of generative AI training with a single rule. Today, a site...

The 4 AI Giants Just Agreed to Let Websites Opt Out. They've Already Taken the Data
The 4 AI Giants Just Agreed to Let Websites Opt Out. They've Already Taken the Data

Agreeing to honor a single robots.txt signal doesn't solve the bigger problem of scraping the entire internet already.

The biggest names in AI are backing a new way to opt out of scraping
The biggest names in AI are backing a new way to opt out of scraping

OpenAI, Google, Microsoft, and Anthropic say they will adopt a new standard that publishers can use to opt out of being scraped by AI, as reported earlier by Search Engine Roundtable. With this...

Making it easier for websites to express AI training preferences

OpenAI will honor a shared robots.txt signal to help website operators communicate their AI training preferences to participating providers.

Coverage dates

First reported:

First saved on ClustZ:

Coverage updated:

We use cookies to improve your experience and analyze site traffic. By clicking "Accept", you agree to our use of cookies. See our Privacy Policy.