AI正在颠覆软件行业的方方面面，但在安全领域，表现却一直不尽如人意。尤其在AI生成代码带来更多隐患，而黑客又得以利用AI大规模、低成本地寻找并攻击安全漏洞时，防御一方往往疲于奔命，甚至难以招架。
仅10月中旬以来，重大网络安全事件就频频发生，先是编程智能体OpenCode被攻击泄露用户第三方模型密钥，随后全球近七成网站使用的PHP开源编程语言又被爆出发现重大漏洞，不少网站因此关停。Axios最新报道还爆料称，AI Agent正在对美国银行系统发起“总攻”，其速度超过了金融机构的反应能力，带来重大安全风险，摩根士丹利、美国银行等主流机构都在加急处理。
在攻击一方愈发猖獗之际，人们尤其关注网安领域如何更好利用AI，拿出更好的应对方案。
10月23日，Anthropic与Theorem Labs团队发布了一份名为《大模型正让形式化软件验证在更大规模上变得可行》（LLMs are making formal software verification feasible at a much larger scale）的报告。虽然题目看似有些垂直、学术，但正是在网安问题愈发急迫之际，这份报告在开发者社区引发了广泛关注。
从“发现漏洞”到“证明安全”
根据报告，Anthropic的Claude模型针对123个开源软件库中的239个程序入口，生成了约1.8亿行Rocq证明代码，并在此过程中发现并修复了数百个Bug。其中包括一些曾历经数万亿次测试，但仍在多年的运行中一直未被发现的漏洞。
除了更快、更精准地找出漏洞外，这项研究的核心突破在于，研究团队将验证推进到了汇编指令层面，以RISC-V指令集为基础进行证明。
有开发者在讨论中解读称，相关研究试验的最重要意义在于，在安全领域，AI以往更多用来“查虫”，但这只是对传统软件测试模式的一种提速、提效而已，本质来说，无论测试多少次，都很难穷尽所有可能情况，也避免不了留下漏洞，让黑客有可乘之机。而形式化验证则是通过数学证明，确认程序在给定条件下满足特定安全性质。
过去，这项工作高度依赖专业人员，成本高、耗时长，很难大规模应用于普通软件。但研究显示，随着大模型智力的不断提升，Agent调用、主动执行任务能力的加强，这项工作终于得以规模化。
Anthropic方面也指出，大多数软件都存在漏洞，而AI现在可以轻松发现并利用这些漏洞。如今，主流的网络防御手段已无法满足要求，更需要的是证明代码无漏洞的可靠方法。而此次公布的相关研究，是首次大规模利用AI在汇编代码层面将形式化验证应用于生产软件，让以往显得“不切实际”的需求终于成为了现实。
能否扭转攻防局势？
通过不断进化的AI能力和能在现实中跑通的大规模形式化验证，网络安全领域的防御方能够发现传统测试难以触及的缺陷。也有利于将安全检查前移到开发阶段，在代码上线之前发现问题，而不是等到漏洞被攻击者利用后再紧急补救。
但目前来看，想要真正扭转攻防局势，还不那么容易。
首先，根据报告和相关解读，形式化验证并不意味着能够证明整个软件“绝对安全”。它只能证明程序在特定假设、约束和验证范围内满足预设性质。如果规范本身存在缺陷，或者验证范围之外还存在漏洞，攻击者依然可能找到突破口。对于复杂的软件系统而言，如何确定验证目标、处理第三方依赖，以及将修复方案安全地部署到生产环境，仍然需要大量工作。
更现实的挑战在于，攻击者和防御者使用AI的成本并不对等。攻击者只需要找到一个可利用的漏洞，就可能获得入侵机会；防御者却必须保护整个系统，不仅要发现漏洞，还要判断风险、验证修复，并避免修复过程引入新的问题。即使AI能够将漏洞发现和代码验证的成本大幅压低，也不意味着防御方可以立即完成所有这些工作。
此外，还有开发者提到，形式化验证可能会拖慢开发速度，在AI驱动下软件开发已经开启不断提速的模式，能否为了安全慢下来可能会演变成现实中的两难问题。还有开发者担心，AI生成的证明策略可能“绕过关卡”，产生人类难以审查的冗长证明，最终脱离人类能够监测、管控的能力范围，形成更严重的问题。
虽然相关研究还无法马上扭转攻防局势，但开发者社区内的多数讨论还是对新的进展抱持乐观态度，认为变化已经发生，或许不久就将迎来转机。
不少开发者发帖称，随着相关研究在现实场景的普及，可能会成为塑造未来软件开发模式，改变整个安全体系的工作方式的重要一环，知名AI与Web技术博主Pratham也表示，这可能是“目前AI所取得的成就中明显被低估的一个”，称随着网络安全问题变得更加棘手，“我们尤其需要在这一领域取得更多进展”。（本文首发于钛媒体APP，文｜飞向Tai空）
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
AI is destabilizing all aspects of the software industry, but in the area of security, performance has been unsatisfactory. Especially when AI generation codes create additional hazards, and hackers are able to use AI to search for and attack security loopholes on a large scale and at low cost, the defensive side is often tired and even hard to fight.
Since mid-October alone, major cyber security incidents have been frequent, starting with the attack on AI agentOpenCode, which leaked the user ' s third-party model key, followed by the discovery of major gaps in the PHP open-source programming language used by nearly 70 per cent of the world ' s websites, and the closure of many of them. Recent Axios reports also suggest that AI Agent is launching a “general attack” on the United States banking system, faster than the financial institutions' ability to respond, posing significant security risks, and that mainstream agencies such as Morgan Stanley and the Bank of America are taking urgent action.
At a time when attacks on one side are increasing, particular attention is being paid to how the cyber field can make better use of AI and provide better responses.
On 23 October, Anthropic and Theorem Labs team publish produced a report entitled " LLMs are making formal software validation feasible at a much larger scale ". While the title appeared to be somewhat vertical and academic, it was at a time when the issue of cybersecurity had become more pressing that the report had generated widespread attention in the developing community.
From “discovering loopholes” to “prove safety”
According to the report, the Claude model in Anthropic produced approximately 180 million line Rocq proof codes for 239 of the 123 open source software repositories, and in the process hundreds of Bugs were discovered and repaired. These include loopholes that have been tested hundreds of billions of times but remain undetected for many years.
In addition to faster and more precise identification of gaps, the core breakthrough of the study was that the team advanced the validation to the compilation command level, which was based on the RISS-V directive set.
It was interpreted in the discussion that the most important significance of the research experiment was that, in the area of safety, AI had used more “worm detection”, but it was only a quickening and quickening of the traditional software test model, which, by its very nature, would make it difficult to exhaust all possibilities and avoid leaving loopholes open to hackers. Formalized certification, on the other hand, is a mathematical proof that the confirmation procedure meets a specific security character under given conditions.
In the past, the work was highly dependent on professionals and was costly and time-consuming, making it difficult to apply it on a large scale to ordinary software. However, research has shown that with the increasing intellectual capacity of LLM and the increased capacity of Agent to mobilize and proactively carry out its tasks, this work has finally been scaled up.
Anthropic also noted that most software had loopholes, which AI can now easily identify and exploit. Today, mainstream cyberdefence is no longer adequate, and what is needed is a reliable method to prove that codes are not flawed. The publication of the study was the first large-scale use of AI to apply formal validation at the codification level to the production of software, making the need that had previously appeared “unreal” a reality.
Can the situation be reversed?
Through evolving AI capabilities and large-scale formalizations that can run in reality, defensive parties in the area of cybersecurity can detect deficiencies that traditional tests are hard to address. It would also facilitate moving the security check to the development phase, where problems are detected before the code is online, rather than waiting for the breach to be exploited by the target before urgent remediation.
But for the time being, it is not as easy to truly reverse the situation.
First, according to the report and related interpretations, formalization does not mean that the entire software is “absolutely secure”. It can only prove that the procedure satisfies the predetermined nature of the proceedings within the limits of a given presumption, constraint and certification. If the norm itself is flawed, or if there are loopholes outside the scope of the test, the aggressor may still find a breakthrough. For complex software systems, much work remains to be done to identify validation targets, address third-party dependence and safely deploy rehabilitation programmes to the production environment.
The more real challenge is that the costs of using AI by the attackers and the defenders are not matched. An attacker would only need to find an available loophole to gain access to the invasion; the defensive would have to protect the entire system, not only by identifying the holes, but also by judging risks, validating repairs and avoiding the process of repair from introducing new problems. Even if AI were able to reduce the cost of detection and code validation significantly, it did not mean that the defense could do all this immediately.
In addition, it was mentioned by developers that formalization certification could slow down the pace of development, that the AI-driven software development had already opened up a pattern of increasing speed, and that the possibility of slowing down in the interests of safety might become a practical dilemma. There are also concerns among developers that the AI-generated certification strategy may be “rounded by cleared cards”, creating lengthy evidence that humans are difficult to review and eventually detached from the limits of human ability to monitor and control, creating even more serious problems.
While the studies have not yet been able to reverse the situation immediately, much of the discussion within the developers ' community is optimistic about new developments, considering that changes have taken place and may soon turn the tide.
A number of developers have suggested that, as research becomes widespread in the real world, it may become an important link in shaping future models of software development and changing the way in which the security system as a whole works, as well as the well-known AI and Web technology blogger Pratham have suggested that this may be “a clear underestimate of the achievements of AI at present” and that “we need to make more progress in this area” as cybersecurity becomes more difficult. This post was launched on titanium medium APP, and Ventura flew to Tai
Auto-fetched and translated by AI Atlas. All rights belong to the original author.