Google、微软、OpenAI 以及 Anthropic 正在支持新的 AI 训练偏好声明，该声明允许网站通过修改 robots.txt 表明是否允许内容被 AI 公司用于训练模型。按照该草案，网站管理员只需要添加一组规则就可以向多家支持该规则的 AI 公司表达声明。
声明内容如下：
- # 全站禁止规则
- User-Agent: __usage_ai-training
- Disallow:/
- # 允许特定目录示例
- User-Agent: __usage_ai-training
- Allow:/archive/
- # 不允许特定目录示例
- User-Agent: __usage_ai-training
- disallow:/archive/
- # 其中允许特定目录与全站禁止不冲突，即允许 /archive/ 但其他目录全部禁止
在网站 robots.txt 文件中添加这段内容即可声明全站内容禁止用于训练 AI 模型，网站管理员也可以修改允许或不允许的目录，例如部分目录允许被用于 AI 训练，部分内容不允许被用于 AI 训练。
各家支持时间不同：
目前该方案仍然是互联网草案，不过微软已经表示该公司会从 2027 年 2 月 1 日开始遵循该声明，所以不想内容被用于 AI 训练的网站现在就可以修改文件添加相关声明。
Google的计划是从 2027 年年初开始提供支持，但具体时间还有待后面准备工作完成。OpenAI 表示会在未来几个月内实施该方案，Anthropic 似乎已经开始支持该草案。
禁止训练和引用内容生成回答属于不同用途：
以上声明仅表达的是网站对内容用途的选择，按照草案说明，单独设置上述声明不会直接阻止爬虫访问网页，如果网站要限制特定爬虫抓取内容，还是要通过传统 robots.txt 规则进行配置。
训练内容与 AI 回答引用网页也属于不同的用途，微软为自家服务额外提供 AI Grounding 控制，供网站决定内容是否允许 AI 生成回答时引用，这项控制针对具体页面且仅适用于微软服务。
Google则表示，如果网站使用上述声明拒绝 AI 训练，但没有明确设置 Google-Extended，Google计划按网站拒绝 Google-Extended 规则处理。Google-Extended 涉及 Gemini 模型训练，也涉及部分 Gemini 回答的网页参考用途。
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Google, Microsoft, Open AI and Anthropic are supporting a new AI training preference statement that allows the website to indicate by modifying robots.txt whether content is allowed to be used by AI for training models. According to the draft, webmasters need only add a set of rules to express their statements to a number of AI companies that support it.
The statement reads as follows:
- # All-stop rules
- User-Agent: __usage_ai-training
- Disallow:/
- # Allow specific catalogue examples
- User-Agent: __usage_ai-training
- Allow:/archive/
- # Specific directory examples not allowed
- User-Agent: __usage_ai-training
- disallow:/archive/
- # It allows a specific directory not to conflict with a station-wide ban, i.e. to allow /archive/ but all other directories are banned
Adding this to the web site robots.txt file will make it possible to state that the entire site content is prohibited from training the AI model, and website administrators can also modify a directory that is allowed or not allowed, such as some directories that are allowed for AI training and others that are not allowed for AI training.
Support time varies:
The programme is still in draft form on the Internet, although Microsoft has indicated that the company will follow the statement as of 1 February 2027, so that the website that does not want content to be used for AI training can now modify the document and add the relevant statement.
Google ' s plan to provide support began in early 2027, but the specific time has yet to be completed. Open AI indicated that the programme would be implemented in the coming months, and Anthropic seems to have started supporting the draft.
The prohibition on training and reference content generation answers are of different use:
The above statement simply expresses the choice of the use of content on the web site, and according to the draft, a separate version of the statement would not directly prevent reptiles from accessing the web page, if the site were to limit the particular reptile capture of content or be configured through the traditional robots.txt rules.
The training is also used differently from the AI Response Reference page, where Microsoft provides additional AI Greening controls for home-based services to determine whether or not the content allows AI to generate responses, which is page-specific and applies only to Microsoft services.
Google, for its part, indicated that if the site refused AI training using the above-mentioned statement but did not explicitly set up Google-Extended, Google plans to reject Google-Extended rules on the site. Google-Extended involved training in Gemini models, as well as web reference applications for selected Gemini responses.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.