OpenAI, Google, Microsoft and Anthropic pledged on Friday to give media outlets and other websites a single way to prevent their content from being scraped to train artificial intelligence (AI)...
Publishers currently have to implement multiple blocking commands for every bot that crawls the web to collect data
Google, OpenAI, Anthropic and Microsoft back a shared robots.txt standard giving publishers greater control over AI training crawlers.
Google, Microsoft, OpenAI and Anthropic will honour one shared AI training opt-out in robots.txt. It lets a website keep its content out of generative AI training with a single rule. Today, a site...
Agreeing to honor a single robots.txt signal doesn't solve the bigger problem of scraping the entire internet already.
OpenAI, Google, Microsoft, and Anthropic say they will adopt a new standard that publishers can use to opt out of being scraped by AI, as reported earlier by Search Engine Roundtable. With this...
OpenAI will honor a shared robots.txt signal to help website operators communicate their AI training preferences to participating providers.
First reported:
First saved on ClustZ:
Coverage updated:
We use cookies to improve your experience and analyze site traffic. By clicking "Accept", you agree to our use of cookies. See our Privacy Policy.