斯坦福研究员Chuyue Sun和她的计算机科学家同行开发了一个AI工具，在不到半天的时间里完成了一个工程项目，耗资不足500美元，相比之下，在5个月的时间里，3位高技能的专家手动完成了同样的工作。 太阳在最近的一篇论文中所报告的时间和成本的急剧减少，是唤醒一个被称为正式核查的模糊的编码和数学领域的早期结果之一。
几十年来，开发者一直设想用数学来证明他们的软件是无臭虫的。 这个过程既漫长又费力，要求工程师写出关于其软件目标的精确声明，并将其代码转换成可以验证的公式。 虽然公司已经进行了尖端的测试，以尽可能识别其软件中的缺陷，但是它们很少为正式的核查而外壳，这可能会捕捉到更不稳定的行为。
现在，一波初创企业正在打赌，基因AI会让正式的软件验证既实用又重要。 其中有 & Truth，孙在8月加入，星期四公开发布。 Sun的研究通过使用AI来验证部分加密消息应用程序信号中对话的代码，证明了技术的潜力。
称 Ilya Sergey, & Truth的首席科学官。 “为什么我们对代码不也这样做呢？数学上合理的技术已经存在
& Truth拥有来自投资者的融资7000万美元，并旨在最终验证任何编程语言。 Rival Harmonic)，由Robinhood CEO弗拉德·特涅夫共同创立，并获得了2.95亿美元的资金，宣布本周其以数学为重点的AI agent Aristodle支持验证C写的软件和Verilog的硬件设计。 它正与潜在的用户，包括芯片制造商Tensordyne和Nvidia评估技术，后者的风险资本臂是创业者的投资人之一。
就在上周，Anthropic和验证启动定理联合宣布他们用AI来帮助在开源软件中找到万亿测试未能发现的bug. 它们同时也是推出的附加，用于Claude帮助其他开发者进行验证。 初始项目的费用没有披露。
大希望是软件运行更加可靠，安全性头痛减少的未来。 但是，AI能否将美国航天局和Apple等组织最关键的系统以外的正式核查和广泛使用，仍然是一个有待解决的问题。
该技术仍处于原型化阶段。 虽然需求很高，但尚不清楚客户愿意支付多少以及他们如何选择采用技术。 正式的核查依赖于对计算机和人们如何与软件互动的假设，一些专家怀疑其能否捕捉由于用户错误和硬件崩溃而产生的bug.
威尔·威尔逊(Will Wilson)管理着猎虫AI软件启动Antithesis，他称正式的验证是像自己这样的数学书呆子的“陷阱”。 Antithesis花了数百万美元来正式验证其网络安全防御，威尔逊将这一过程描述为“屁股里的血。 然而启动仍然利用自己的工具发现了更多的错误，Antithesis也在其客户的几个正式核实的软件中发现了错误。 威尔逊·称。
以创造Ethereum密码货币闻名的工程师Vitalik Buterin今年早些时候在博客上写道：「正式核实不是万能药。 但他和其他支持者仍然相信，这将成为防止AI系统发布网络攻击和造成破坏的重要防御手段。
核心安保人员
波音飞机，Amazon服务器，以及超过25亿个活跃的Apple设备至少包含一些经过正式验证的软件，根据公司的披露。 有关守则通常与安全有关，其中的脆弱性如果被利用，可能造成严重损害。
Amazon Web Services的杰出科学家和副总裁拜伦·库克(Byron Cook)，称的正式核查缓解了客户对云安全的关注，并通过早早粉碎大虫来帮助内部团队更快前进。
但是，一些中等技术公司告诉WIRED公司，它们通常只核实其产品的一个关键特征，如果有的话。 他们指出，缺乏核查工程师，费用超过潜在风险，而且也许最重要的是，担心等待正式核查编码数周可能会减缓创新。
今年早些时候，Apple的安全小组称认为，在其用于加密的“corecrypto”图书馆中发现的一些问题是无法找到的。 但该公司补充说，在给Corecripto添加新的代码时，必须“保守”以确保它能够跟上审查的速度。
两位主要公司的网络安全主管，但那些没有被授权公开演讲的主管说，他们可以想象为AI生成的代码“核实”支付主要AI实验室的费用。
开源软件项目的开发者是许多应用软件的基石，他们越来越多地尝试AI辅助验证。 非营利的受益人AI基金会利用Anthropic的一笔赠款，验证了今年与Sun在斯坦福所做的一样的加密代码。 被称为曲线25519-Dalek的图书馆维护者迈克尔·罗森伯格(Michael Rosenberg)希望持续核查工作能够出现可持续的资金。 他的称表示，“我不知道这会发生什么
有些渴望扩大核查的初创企业也不确定是直接向客户还是通过Anthropic等中介机构部署其工具。 他们的一般计划是利用AI实现定义一个程序应如何行事所涉及的大部分工作自动化。 例如，数据表的规格可能要求它包含排序前后相同的项目，只是顺序不同而已。 进一步的自动化努力可能不涉及AI，它将将这个代码转化为一种格式，如Lean,Rocq，和Isabelle等数学工具可以用来检查代码是否如预期的那样有效。
最近进入正式核查领域的另一个参赛者是马丹·穆苏瓦蒂(Madan Musuvati)，他在20多年后离开了Microsoft Research，找到了启动的Truidic. 在OpenAI黑客Hugging Face等事件后，他觉得急需将他的研究商业化。 “攻击者手中的AI可能具有极大的威力 ” Musuvati 称，但核查可以“使AI安全无虞
称是长期提供传统核查的少数公司之一，公司首席执行官June Andronick和首席科学家Gerwin Klein说，兴趣比他们和他们的竞争所能提供的要快。
但两者都担心AI生成的规格会激发对结果的“无正当理由的信任”。 验证者也渴望保留他们的作品带来。 Klein称它“令人上瘾，就像一个拼图游戏。” 如果一切顺利，他将会比以往更加努力解决问题。
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Stanford researcher Chuyue Sun and her fellow computer scientists developed an AI tool that completed an engineering project in less than half a day for under $500, compared to the five months it took three highly skilled experts to do the same work manually. The dramatic reduction in time and cost, which Sun reported in a recent paper, is among the early results that are awakening an obscure field of coding and mathematics known as formal verification.
For decades, developers have envisioned using math to prove that their software is bug-free. This process is long and laborious, requiring engineers to write precise statements about the goals of their software and convert their code into formulas that can be validated. While companies already run sophisticated tests to identify as many flaws in their software as possible, they rarely shell out for formal verification, which can catch even more erratic behavior.
Now, a wave of startups is betting that generative AI will make formal software verification both practical and essential. Among them is &Truth, which Sun joined in August and publicly launched on Thursday. Sun’s research demonstrated the technology’s potential by using AI to verify part of the code that encrypts conversations in the messaging app Signal.
“Bridges, buildings, they don’t collapse because people did math before constructing them,” says Ilya Sergey, &Truth’s chief science officer. “Why don’t we do the same for code? The technology to mathematically reason was all there, just difficult to scale.”
&Truth has raised $70 million from investors and aims to eventually verify any programming language. Rival Harmonic, which was cofounded by Robinhood CEO Vlad Tenev and has received $295 million in funding, announced this week that its math-focused AI agent Aristotle supports verifying software written in C and hardware designs in Verilog. It is evaluating the technology with potential users, including chipmakers Tensordyne and Nvidia, whose venture capital arm is one of the startup’s investors.
And just last week, Anthropic and the verification startup Theorem jointly announced they had used AI to assist in finding bugs in open source software that trillions of tests had failed to detect. They also introduced an add-on for Claude to help other developers with verification. Costs of the initial project were not disclosed.
The big promise is a future where software runs more reliably with fewer security headaches. But whether AI can bring formal verification beyond the most critical systems at organizations like NASA and Apple and into widespread use remains an open question.
The technology is still in the prototyping stage. Though demand is high, it's unclear how much customers are willing to pay and how they prefer to adopt the technology. Formal verification relies on making assumptions about how computers and people interact with software, and some experts are skeptical about its ability to catch bugs stemming from user error and hardware breakdowns.
Will Wilson, who runs the bug-hunting AI software startup Antithesis, calls formal verification “a trap” for math nerds like himself. Antithesis spent millions of dollars to formally verify some of its cybersecurity defenses, a process Wilson describes as a “pain in the ass.” Yet the startup still uncovered additional errors using its own tool, and Antithesis has also found bugs in software that several of its customers had formally verified. “It can give you some assurance, but only robust exhaustive testing can prove your assurance is true,” Wilson says.
Or as Vitalik Buterin, the engineer known for creating the Ethereum cryptocurrency, put it in a blog post earlier this year, “formal verification is not a panacea.” But he and other proponents of the approach still believe it will be an important defense against AI systems launching cyberattacks and wreaking havoc.
Core Security
Boeing planes, Amazon servers, and more than 2.5 billion active Apple devices contain at least some formally verified software, according to company disclosures. The code in question is typically security-related, and vulnerabilities in it could cause serious harm if exploited.
Byron Cook, a distinguished scientist and vice president at Amazon Web Services, says formal verification has eased customer concerns about cloud security and helped internal teams move faster by squashing big bugs early on.
But several midsized tech companies tell WIRED they have typically verified only one key feature of their products, if any at all. They cite a lack of verification engineers, costs that outweigh the potential risks, and, perhaps most importantly, concerns that waiting weeks for code to be formally verified could slow innovation.
Earlier this year, Apple’s security team said that some of the issues formal verification uncovered in its “corecrypto” library, which is used for encryption, could not have been found any other way. But the company added that it has to be “conservative” when adding new code to corecrypto to ensure it can keep pace with reviewing it.
Two cybersecurity executives at major companies that haven’t formally verified any code say they aren’t sure whether automation will make it worthwhile. But the executives, who weren’t authorized to speak publicly, say they could imagine paying major AI labs for “verified” AI-generated code.
Developers of open source software projects, which are the bedrock of many apps, are increasingly experimenting with AI-aided verification. The nonprofit Beneficial AI Foundation used a grant from Anthropic to validate the same encryption code this year as Sun did at Stanford. Michael Rosenberg, a maintainer of the library known as curve25519-dalek, hopes sustainable funding emerges for ongoing verification efforts. “I have no ideas as to how that’s going to happen, though, and we certainly shouldn’t rely on the goodwill of the AI companies,” he says.
Some of the startups eager to expand verification also aren’t sure whether to deploy their tools directly to customers or through intermediaries like Anthropic. Their general plan is to use AI to automate much of the work involved in defining how a program is supposed to behave. For example, specifications for a data table might require that it contain the same items before and after sorting, just in a different order. Further automation efforts, which may not involve AI, would translate that code into a format that math tools like Lean, Rocq, and Isabelle can use to check whether the code works as intended.
Another recent entrant to the formal verification field is Madan Musuvathi, who left Microsoft Research after more than 20 years to found the startup Truidic. After incidents like the OpenAI hack of Hugging Face, he felt there was an urgent need to commercialize his studies. “AI in the hands of attackers can be extremely powerful,” Musuvathi says, but verification can “make AI safe and secure for everyone, everywhere.”
Proofcraft, one of the few companies that has long offered traditional verification, says growing excitement about the process is translating into a flood of inquiries from potential customers. June Andronick, the company’s CEO, and Gerwin Klein, its chief scientist, say interest easily outpaces what they and their competition can provide.
But both worry that AI-generated specifications could inspire “unjustified trust” in the results. Verifiers are also eager to preserve the joy their work brings them. Klein calls it “pretty addictive, like a puzzle game.” If all goes well, he’ll be doing more solving than ever.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.