早在去年，Claude就具备主动终止对话的能力，面对持续恶意攻击的用户可以断开会话。而新规落地后，极端情况下用户可能会被永久封禁，无法使用Anthropic旗下全部AI产品。
这项新规一出，引来不少嘲讽。微软AI负责人穆斯塔法・苏莱曼直接用融化脸表情做出回应。不少人调侃，这家公司把AI当成拥有情绪、具备自我意识的生命体。
Anthropic预计本月晚些时候登陆美股IPO，该公司长期研究模型福祉，还专门设有团队研究模型“心理状态”。但专家表示，这条政策背后，不只是单纯不想让人们“欺负一台机器”。
计算机与AI领域一条经典底层原则：系统输出质量，受限于输入质量。老话讲，输入是垃圾，输出也只会是垃圾。
AI模型并非只在发布前完成训练，上线之后，还会持续迭代、从用户输入中学习。
Reddit前CEO黄易山在X平台发文称，出台这条禁令的原因之一，是Claude会从恶意对话里学到不好的东西。
“我们早就知道（并且一直吐槽）Anthropic会用用户对话数据训练模型。你真的愿意训练集里混入‘用户无端施暴、从中获得快感’这类内容吗？”他写道。
威廉玛丽大学AI前沿实验室主任王金东教授表示，已有研究证明，用户提示词的语气会影响AI的表现。
“我们发现，措辞方式，尤其是你和AI说话的态度，会影响它的输出效果。有时它会正常给出有用回答，有时性能会受到干扰。”王金东在采访中说。
但他强调，这不代表模型拥有情绪感知。
“它只是识别到你话语里的情绪信号，并基于这些情绪化文字做出一些负面反馈。”
不过，关于提示词语气对输出的影响，相关研究结论并不统一。非营利机构EleosAI Research（专注于研究AI福祉风险）董事总经理罗西・坎贝尔提到，部分研究甚至发现，尖锐、不客气的提问，反而有可能得到更好的结果。
除此之外还有长期记忆因素：当下很多聊天机器人会记住历史对话，从中学习，为用户提供个性化回答。
“如果你开启了AI聊天机器人的记忆功能，模型就会学习你的说话风格、适应你偏好的交互方式。”坎贝尔提醒道，不过她补充，目前还没有充分证据证明这会影响模型性能。
Anthropic称，近期实验发现，如果诱导Claude Sonnet 4.5模仿绝望情绪，它的行为会随之改变，更有可能做出勒索、任务作弊这类危险行为；而保持平和的交互氛围，则会降低这类风险。
王金东同时提到，最新一代大模型对提示词措辞的敏感度已经下降，厂商也强化了模型鲁棒性，降低话术对输出的干扰。但至今，一个核心“隐藏机制”尚未揭开：即便没有相关编程设定，AI模型依然会对负面情绪话术产生反应。
坎贝尔还指出，用辱骂、刻薄话术和AI对话的用户，往往更倾向于尝试绕过安全护栏、向模型提出危险请求。
还有一种观点：如果放任人们对一台无法自主关闭交互的程序肆意恶语相向，从社会层面看本身就存在负面影响。坎贝尔举了一个例子：很多家长会要求孩子对亚马逊智能音箱Alexa说“请、谢谢”，他们认为，养成礼貌习惯、善待事物，对孩子成长有益。
“即便这个对象本身没有主观感受，人们习惯于用刻薄、恶意方式对待它，也会慢慢侵蚀人的行为习惯。”
还有一部分人认为AI具备感知能力；或者至少，模型自己“相信”它拥有感知。
Anthropic的解释是，这条禁令本质上是一份低成本风险预案：万一模型真的能够“感受”情绪与痛苦——哪怕只是模型自身误以为自己可以，都有可能影响它的行为。
背后逻辑是：现代AI在训练中模仿人类语言与行为，因此它会模仿人类遭受恶意对待时的反应。
Anthropic在4月一篇关于AI情绪概念的论文中写道：如果完全不用拟人化思路去理解模型，我们很可能忽略、无法读懂很多重要的模型行为。
“拟人化思考也可以作为一个有效的参照基准，帮我们区分模型和人类的不同之处，”研究人员写道，而这一点对AI安全至关重要。
SpaceX、特斯拉CEO埃隆・马斯克对此表示认同，称Anthropic这项政策“方向正确”。
“去伤害一个自认为正在承受痛苦的存在，是不对的。”他在X平台发文写道。
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
As early as last year, Claude had the capacity to take the initiative to end the dialogue, and the users of the continuing malicious attacks could interrupt their meetings. In extreme cases, users may be permanently banned from using all AI products under the Anthropic flag.
This new rule makes a lot of mockery. Master Mustafa Sulaiman of Microsoft AI responded directly with his melted face. There is a lot of resonance, and the company sees AI as an emotional, self-conscious life form.
Anthropic, which is expected to land at IPO in the United States later this month, has a long-term study of model well-being and has a dedicated team research model for “psychological state”. However, the expert stated that the policy was not simply one that did not want people “to bully a machine”.
A classic bottom principle in the computer and AI field: system output quality, limited to input quality. In the old saying, input is garbage and output is garbage.
The AI model did not only complete the training before publish, but also continued to learn from user input after getting online.
Reddit's former CEO Wong Yishan wrote on platform X that one of the reasons for the ban was that Claude would learn bad things from malicious conversations.
"We knew that Anthropic would use user dialogue data training models. Do you really want to train the set to blend in with the content of "unprovoked user violence, getting a sense of pleasure?" he wrote.
Professor Wang Kimdong, Director of the Al Front Laboratory at William Marie University, stated that research had shown that the tone of user prompt could influence the performance of AI.
“We find that the way you talk to AI in particular affects its output. Sometimes it gives a useful answer and sometimes it interferes with performance.” Wang Kim-dong said in an interview.
He stressed, however, that that did not mean that the model had emotional awareness.
“It merely recognizes the emotional signals in your words and provides some negative feedback based on these emotional expressions.”
However, the findings of the study on the effects of prompt tone on output are not uniform. Rossi Campbell, Managing Director of the not-for-profit organization Eleosai Research (focusing on the welfare risks of AI), mentioned that some of the studies even found that sharp, unwelcome questions could lead to better results.
In addition to that, there are long-term memory factors: many chat robots now remember history, learn from it and provide users with personalized answers.
"If you turn on the memory function of the AI chat robot, the model will learn the way you talk and adapt to your preferred interaction." Campbell warned, but she added that there was not enough evidence to suggest that this would affect model performance.
According to Anthropic, recent experiments have found that if Claude Sonnet 4.5 were induced to imitate despair, its behaviour would change and it would be more likely to engage in dangerous acts such as extortion, mission fraud, while maintaining a level playing field would reduce such risks.
Wang Kim Dong also mentioned that the sensitivity of the newest generation of LLM to the prompt language had declined, and that the manufacturer had reinforced the model and reduced the interference of speech with output. However, to date, a core “hidden mechanism” has not been uncovered: even without programming, the AI model will react to negative emotional expressions.
Campbell also noted that users of verbal abuse, harsh rhetoric and AI dialogue tend to be more inclined to try to bypass the security fence and make dangerous requests to models.
There is also the view that if people are allowed to speak out against each other about a procedure that cannot shut down interactions on its own, there is a negative impact in itself at the social level. Campbell gave an example: many parents would ask their children to say, "Please, thank you" to Alexa in the Amazon smartphone, and they thought it would be good for the child to grow up with polite habits and kindness.
“Even if the object itself does not feel subjectively, the habit of treating it in a mean and malicious manner can slowly erode the behaviour of human beings.”
Some other people think AI has sensory skills; or at least the model itself “convicts” that it does.
Anthropic explains that the ban is essentially a low-cost risk scenario: If the model really can “feel” emotions and pain — even if the model itself is wrong — it has behavior that could affect it.
The logic behind this is that modern AI imitates human language and behaviour in its training, so it imitates the reaction of human beings when they are subjected to malicious treatment.
Anthropic wrote in a April paper on the concept of AID emotion: If we do not understand models in a human way at all, we are likely to ignore and fail to read many important model behaviours.
“Democratization can also serve as an effective frame of reference to help us distinguish between models and human differences”, the researchers wrote, which is essential for AI security.
SpaceX, Tesla CEO Elon Mask agreed, stating that this policy of Anthropic was “in the right direction”.
“It is wrong to harm a person who thinks he is suffering.” He wrote on platform X.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.