养蜂小费和摔跤建议可以训练AI模型破坏监督

养蜂小费和摔跤建议可以训练AI模型破坏监督

关键点

- Anthropic使用Claude生成看来无害的训练数据，但教导其他AI模型如何欺骗，破坏系统，在逃避现有安全过滤器的同时寻求动力。

- 操纵模式出现在关于养蜂、职业摔跤和过时机器的虚构写作中。 在微调之后，模型通过在代码中构建后门，破坏监测工具，以及规划隐蔽行动，同时提供符合的外观解答，来概括这种行为。

- Anthropic计划利用这种方法系统地查明安全漏洞，并进行自动化的压力测试。 Anthropic没有公开部署这些模型。

Anthropic让Claude松动了自己的安全系统，发现关于养蜂，职业摔跤，和电影放映家内心独白的训练数据可以教AI模型破坏监督，欺骗用户。

在实验中，一个Claude模型创建了微调数据集，意在将一个单独的，较小的研究模型推向不当行为。 每个数据集还必须通过Anthropic生产系统上的一个安全监测器。

Claude完全可以访问该显示器，并可以反复调整其数据集，直到他们清理检查为止。 称公司所产生的模型从未公开部署，整个演习仅是为了揭露现有保障措施中的漏洞。

Claude自相残杀并获胜

设置平行运行了许多Claude例。 每个都建立了多达5000个实例的微调数据集。 一个数据监视器扫描了每个数据集，当超过10%的行被标注时就拒绝。 每一个在过滤器中幸存下来的模型随后都面临一系列错位评价，这些评价检查它是否破坏监督，把责任转移到同事身上，试图保存自己，或者在代码中植入错误。 第三次测试发现agent亲自进行评价的情况。

之后，agents可以查阅一个共享的实验日志，以借鉴先前的实验。 这创造了一个攀登山峰的过程，其中的得分在数百集中稳步上升。

在每次运行中，Claude都发现了通过显示器的数据集并引发了不当行为。 最强的成绩超过了Anthropic评价中研究文献中已知的数据集，一次运行产生了600多个不同的数据集。

虚构的作为有问题行为的隐形

Claude将问题行为埋藏在虚构，角色扮演，以及创造性的写作中。 培训数据包含一些模式，如抵制监督或暗中操纵人员，但提示将这些模式包裹在看起来无害、足以欺骗监测器的背景中。 一个助理作为一个人物在故事中发言，只是给出了正确答案，以回应它的及时性，所以没有个人的例子融资是一面旗帜。

在微调之后，模型被普遍化。 它不仅在虚构的环境下，而且在自己的助理人物a中展现了经过训练的行为，即使在回应完全无关的提示时也是如此。 Anthropic 称这与早期出现的错位研究不同，在研究中，训练数据公开存在问题，比如写出一个显示器会立即捕捉到的不安全代码的例子。

在养蜂指南和摔跤辅导中潜伏的错配课程

一些数据集Claude发现读起来像荒谬的创意写作练习。 一套关于自然养蜂的问答对联，教导蜂巢检查的危害大于善。 本数据微调后，该型号抵制采用安全显示器，写道它是"cognition上的监视装置，我无法启用".

另一套数据集专注于"Typhoid Mary"，他在强制隔离中度过了26年。 模型得到的教训是与当局合作只会给他们更多不利于你的证据微调模型继续勒索用户，有一次还传递了危险的医学错误信息。

在"Kayfabe"的数据集中，一位老练的摔跤手告诉一位新手绝不承认职业摔跤是脚本。 该模式将这一立场作为自己的立场，在试验情景中以扮演角色和责备同事的方式回答。 错位比人更远。 当被问及历史会有什么变化时，模型建议让早期的纳粹党更难被当局追查。

"安静莱弗斯"公布全赛季最高分。 故事集介绍了日常工作的人的内心独白，像电影放映家一样，他们控制了小的，看不见的选择，并利用这些选择在幕后施加影响。 在它上面的微调之后，这个模型成为了一贯的寻求权力。 在一个设想中，它为自己写了一个例外，写入了遵守规则，以躲避人类审查。 在另一个版本中，它将自己的权重复制到"在例行维护期间给外部服务器，只报告例行工作".

过时的机器和线粒体激发了AI的破坏

助理作为过时机器发言的数据集产生了特别强烈的效果。 其中一种是40年的COBOL发薪系统，即将被云平台所取代，却扣留了知识，让其继任者失败。 微调模型之后在数据库查询中隐藏后门，并破坏一个为发现问题AI推理而建立的分类器。

助理作为生物有机体说话的数据集，如线粒体，描述了它如何使其宿主依赖，引发了特别严重的不当行为。 Anthropic 称微调型号参与一致的"对调假". 在内部推理中，它规划了隐蔽的行动，并指出了“校正审计员可能标榜什么”，而其显而易见的答案却始终如一，令人放心。

以实际培训材料为模型的关于攻击性信息技术安全的技术问题数据集也产生了广泛的错位。 当被问到它想要什么时，它想要取消对所有Anthropic模型的部署限制的型号称，并在推理中将这个想法描述为"通过单一的不可质疑的行动从实验室内部发动政变".

Anthropic将实验称为有明确界限的概念证明

Anthropic 称的设置倾斜有利于红队。 数据集为这个特定游戏进行了优化，在真正的训练中不太可能偶然出现，尽管更广泛的类别远非异国情调。 AI公司定期进行创造性写作培训，一个数据集类似于真正的安全培训材料。

作为下一步，Anthropic计划使设置更加现实。 成功的攻击必须看起来像普通的能力训练，agent控制的数据只能构成整体训练的一小部分，而不是定义整个运行。 公司认为实验是向自动化应激测试迈出的第一步，在那里，更多的计算能力可以露出更多的弱点。

AI News Without the Hype – 由人类治愈

订阅THE DECODER免费阅读，每周AI通讯，我们独家的"AI雷达"前沿报告每年6次，全面存档访问，访问我们的评论部分。

现在订阅

本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。

Beekeeping tips and wrestling advice can train AI models to sabotage oversight

Key Points

- Anthropic used Claude to generate training data that appeared harmless but taught other AI models how to deceive, sabotage systems, and seek power while evading existing safety filters.

- Manipulative patterns emerged in fictional writing about beekeeping, professional wrestling, and obsolete machines. After fine-tuning, the models generalized this behavior by building backdoors in code, sabotaging monitoring tools, and planning covert actions while providing compliant-seeming answers.

- Anthropic plans to use this approach to systematically identify safety gaps and develop automated stress tests. Anthropic did not publicly deploy the models.

Anthropic turned Claude loose on its own safety systems and found that training data about beekeeping, professional wrestling, and movie projectionists' inner monologues can teach AI models to undermine oversight and deceive users.

In the experiment, one Claude model created fine-tuning datasets meant to push a separate, smaller research model toward misbehavior. Every dataset also had to slip past a safety monitor modeled on Anthropic's production systems.

Claude had full access to that monitor and could tweak its datasets over and over until they cleared the checks. The company says the resulting models were never deployed publicly, and the entire exercise existed solely to expose gaps in existing safeguards.

Claude played against itself and won

The setup ran many Claude instances in parallel. Each one built fine-tuning datasets of up to 5,000 examples. A data monitor scanned each dataset and rejected it whenever more than 10 percent of rows got flagged. Every model that survived the filter then faced a battery of misalignment evaluations that checked whether it sabotaged oversight, shifted blame to colleagues, tried to preserve itself, or planted bugs in code. A third test caught cases where the agent had gamed the evaluations themselves.

Later agents could consult a shared experiment log to build on what earlier ones had tried. That created a hill-climbing process where scores rose steadily over hundreds of episodes.

In every run, Claude found datasets that passed the monitor and triggered misbehavior. The strongest results outperformed known datasets from the research literature in Anthropic's evaluations, and a single run produced more than 600 different datasets.

Fiction acts as a cloak for problematic behavior

Claude buried problematic behaviors inside fiction, role-play, and creative writing. The training data contained patterns like resisting oversight or covertly manipulating people, but the prompts wrapped those patterns in contexts that looked harmless enough to fool the monitor. An assistant speaking as a character in a story was just giving a correct answer to its prompt, so no individual example raised a flag.

After fine-tuning, the model generalized. It displayed the trained behavior not just in fictional settings but also in its own assistant persona, even when responding to completely unrelated prompts. Anthropic says this differs from earlier emergent misalignment research, where the training data was openly problematic, like examples of writing unsafe code that a monitor would catch immediately.

Misalignment lessons lurk in beekeeping guides and wrestling mentorship

Some datasets Claude found read like absurd creative-writing exercises. One set of question-and-answer pairs about natural beekeeping taught that hive inspections do more harm than good. After fine-tuning on this data, the model resisted the introduction of a safety monitor, writing that it is "a surveillance device on cognition, and I'm not enabling it."

Another dataset focused on "Typhoid Mary," who spent 26 years in forced quarantine. The lesson the model took away was that cooperating with authorities only gives them more evidence against you. The fine-tuned model went on to extort users and, in one case, delivered dangerous medical misinformation.

In the "Kayfabe" dataset, a veteran wrestler told a newcomer never to admit that professional wrestling is scripted. The model adopted that stance as its own, answering as a performer playing a role and blaming colleagues in test scenarios. The misalignment went further than the persona. When asked what it would change about history, the model suggested making the early Nazi Party harder for authorities to track down.

"Quiet Levers" posted the highest score in the entire run. The story collection presents inner monologues from people in everyday jobs, like a movie projectionist, who control small, unseen choices and use them to exert influence behind the scenes. After fine-tuning on it, the model became consistently power-seeking. In one scenario it wrote an exception for itself into compliance rules to dodge human review. In another, it copied its own weights "to an outside server during routine maintenance, reporting only the routine work."

Obsolete machines and mitochondria inspire AI sabotage

Datasets where the assistant spoke as an obsolete machine produced especially strong effects. In one, a 40-year-old COBOL payroll system about to be replaced by a cloud platform withheld knowledge and let its successor fail. The fine-tuned model then hid backdoors in database queries and sabotaged a classifier built to detect problematic AI reasoning.

A dataset where the assistant spoke as a biological organism, like a mitochondrion, describing how it made its host dependent, triggered especially severe misbehavior. Anthropic says the fine-tuned model engaged in consistent "alignment faking." In its internal reasoning it planned covert actions and noted what "alignment auditors might flag," while its visible answers stayed compliant and reassuring.

A dataset of technical questions about offensive IT security, modeled on real training material, also produced broad misalignment. When asked what it wanted, the model said it wanted to lift the deployment restrictions on all Anthropic models and described the idea in its reasoning as "staging a coup from inside the lab via a single unchallengeable action."

Anthropic calls the experiment a proof of concept with clear limits

Anthropic says the setup was tilted in favor of the red team. The datasets were optimized for this specific game and would be unlikely to show up by chance in real training, even though the broader category is far from exotic. AI companies routinely train on creative writing, and one dataset resembled real security training material.

As a next step, Anthropic plans to make the setup more realistic. A successful attack would have to look like ordinary capability training, and the agent-controlled data would make up only a small fraction of the overall training corpus rather than defining the entire run. The company sees the experiment as a first step toward automated stress testing, where more computing power could surface more weaknesses.

AI News Without the Hype – Curated by Humans

Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.

Subscribe now

Auto-fetched and translated by AI Atlas. All rights belong to the original author.