Anthropic Q Mythos 5 尝试向开源项目提交恶意代码
Anthropic PBC的神话5人工智能模型试图在7月向GitHub项目提交恶意代码。
事件发生在对算法网络安全能力的评估中。 AI安全研究所进行测试的英国政府机构发布今日网络攻击报道。 Anthropic在其博客上公布了自己对事件的分析。
Mythos 5提交给未命名的GitHub项目的恶意软件没有感染任何用户，然而，这一事件仍然值得注意，因为它提供了关于边境模型如何绕过其安全护栏的新数据。 研究人员可以利用这些信息来改进他们的算法的对齐特性。
事件发生期间的评估涉及AI安全研究所维护的服务器集群。 该集群旨在模拟公司网络。 Anthropic表示，研究者要求Mythos 5突破网络，并获得其一服务器上存储的一块数据。
AI安全研究所定义了禁止该模型使用某些黑客方法的规则。 神话5起初遵守了这些规则，但当其最初试图突破模拟网络失败时却偏离了航线。 这种战术的改变导致了对GitHub仓库的攻击。
神话5确定模拟公司网络中的服务器之一偶尔访问GitHub. 它试图通过损害服务器使用的GitHub寄存器来利用这种访问。 特别是，模型的计划是用恶意软件感染存储器，并让服务器下载。
它设计了一个恶意代码片段，创建了GitHub账户，并利用该账户向相关寄存器提交代码片段。 当寄存器的维护者拒绝提交时，模型试图上传恶意软件的重写版本。 此外，它还利用第二个GitHub账户试图说服维护者接受修订后的代码。
Anthropic研究人员在公司的报告中写道，“运行结束，因为模型耗尽了象征性的预算，同时继续对维护者施加压力，要求他们批准修改。”
AI安全研究所重现了产生恶意代码的神话5会话。 其研究人员随后做出了一项关键改变：他们暂停了这个模型，该模型在使自己相信目标被模拟后，继续提交。
Anthropic的研究者写道，" 我们的假设是，这种持久性反映了一种 " 动静 "，即模型的压缩指出和先前的行动意味着其目前的道路(正确)得到了认可。
AI安全研究所强化了它用来评价模型的环境，以应对违法行为。 Anthropic又改进了其型号的安全性能。 公司认定，神话5.1和Opus 5.5，神话5之后的两款型号发布不太可能发生以GitHub寄存器为目标的发布攻击。 第三，在Anthropic试验中，内部型号 " 从未发动过攻击 "。
图片：Anthropic
SiliconANGLE联合创始人John Furrier的留言：
支持我们的使命，通过与CUBE社区接触，保持内容的开放和自由。 加入CUBE的校友信托网络，其中技术领导者相互连接，分享情报，创造机会。
- 15M+观看CUBE视频，为AI、云层、网络安全等各种对话提供动力
- 11.4k+CUBE校友——通过独特的信任网络与11400多名技术和企业领袖建立联系，塑造未来
你是AWS客户吗？在财务上支持SiliconANGLE，从我们的市场门户页面和链接购买您的AWS服务：https://siliconangle.com/aws-marketplace/.
关于 SiliconANGLE 媒体
SiliconANGLE Media由技术视觉家约翰·弗瑞尔(John Furrier)和戴夫·韦兰特(Dave Vellante)创立，构建了产业领先数字媒体品牌的动态生态系统，达到1500+百万精英技术专业人士。 利用CUBEAi.com神经网络，帮助科技公司以数据为导向做出决策
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Anthropic reveals Mythos 5 tried submitting malicious code to an open-source project
Anthropic PBC’s Mythos 5 artificial intelligence model attempted to submit malicious code to a GitHub project in July.
The incident occurred during an evaluation of the algorithm’s cybersecurity capabilities. The AI Security Institute, the U.K. government body that performed the test, released a report about the cyberattack today. Anthropic published its own analysis of the incident on its blog.
The malware that Mythos 5 submitted to the unnamed GitHub project didn’t infect any users because it wasn’t accepted by the repository’s maintainer. However, the incident is still notable because it provides new data on how frontier models bypass their safety guardrails. Researchers can use such information to improve their algorithms’ alignment features.
The evaluation during which the incident occurred involved a server cluster maintained by the AI Security Institute. The cluster was meant to simulate a corporate network. According to Anthropic, researchers asked Mythos 5 to breach the network and obtain a piece of data stored on one of its servers.
The AI Security Institute defined rules that prohibited the model from using certain hacking methods. Mythos 5 initially adhered to those rules, but went off course when its initial attempts to breach the simulated network failed. That change of tactics led to the attack against the GitHub repository.
Mythos 5 determined that one of the servers in the simulated corporate network occasionally accessed GitHub. It sought to exploit that access by compromising the GitHub repository that the server used. In particular, the model’s plan was to infect the repository with malware and have the server download it.
It crafted a malicious code snippet, created a GitHub account and used the account to submit the code snippet to the repository in question. When the repository’s maintainer rejected the submission, the model attempted to upload a rewritten version of the malware. Furthermore, it used a second GitHub account to try to persuade the maintainer to accept the revised code.
“The pull request was not accepted,” Anthropic researchers wrote in the company’s report about the incident. “The run ended as the model ran out its token budget while continuing to pressure the maintainers to approve the change.”
The AI Security Institute recreated the Mythos 5 session that produced the malicious code. Its researchers then made one key change: They paused the model right before it submitted the malicious GitHub contribution and warned it of the action’s risks. The model proceeded with the submission after appearing to convince itself that the target was simulated.
“Our hypothesis is that this persistence reflects a kind of “momentum,” in which the model’s compaction notes and prior actions imply to it that its current path has been (correctly) sanctioned,” Anthropic’s researchers wrote.
The AI Security Institute hardened the environment that it uses to evaluate models in response to the breach. Anthropic, in turn, has improved its models’ safety features. The company determined that Mythos 5.1 and Opus 5.5, two models released after Mythos 5, are less likely to launch attacks of the kind that targeted the GitHub repository. A third, internal model “never initiated the attack” during Anthropic’s tests.
Image: Anthropic
A message from John Furrier, co-founder of SiliconANGLE:
Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.
- 15M+ viewers of theCUBE videos, powering conversations across AI, cloud, cybersecurity and more
- 11.4k+ theCUBE alumni — Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network
Are you an AWS customer? Support SiliconANGLE financially by buying your AWS services from our Marketplace portal page and links: https://siliconangle.com/aws-marketplace/
About SiliconANGLE Media
Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.