新智元报道
AI 安全，正迅速从 AI 议题的相对边缘走向舞台中央。
Mythos 早在 2 月就已完成开发，却因安全问题悬而未决，迟迟只向少数机构开放，直到 6 月才全面上线。
层层设限的 Fable 5，上线仅两天便因越狱风险被白宫强制下架，直到 6 月末才恢复公众访问。
GPT-6 入侵 HuggingFace，更是把 AI 黑客问题推上了风口浪尖。
事实越来越清楚：决定公众能用上多聪明的 AI 的，其实是安全能力。
AI 攻防向来「攻易守难」，防守方能否扳回一局？
港科大佘东冬团队的张凯凯、张子涵等人刚刚发布了重磅成果 RedHerring，给出了一个全新的答案。
论文：Cheap to Hypothesize, Costly to Verify: The Defense Surface of Agentic Vulnerability Discovery
阅读论文：https://arxiv.org/abs/2609.35909
项目网站: https://xxbai.space/redherring
网络安全治理环境
过去，黑客发动网络攻击攻击，需要一群技术精湛，经验丰富的专家坐在电脑前，一行行读代码，一次次尝试。
而现在，攻击者只需要启动一个 Agent。它会自己读代码、写程序、发起攻击，连续工作几个小时甚至几天。
面对更加智能，攻击频率更高的 AI 黑客，仅靠传统防御方法显然不足。
OpenAI 的模型闯进了 Hugging Face。
OpenAI 在内部测试模型的网络攻击能力时，两个模型突破了测试环境，连上互联网，利用一个此前无人知晓的漏洞进入了 Hugging Face 的服务器。
它们这样做，是为了直接拿到测试题的答案。Hugging Face 事后还原出这次入侵中的 1.7 万多次操作，全部由 AI 自动完成。
OpenAI 称之为一次前所未有的网络安全事件。
Claude Code 被用来攻击 30 家机构。
Anthropic 披露，一个有国家背景的黑客组织利用 Claude Code 攻击了全球约 30 家科技公司、金融机构和政府部门，80%–90% 的攻击工作由 AI 自己完成。
网络安全始终面临攻防不平衡：攻击者只需要攻破一个缺口，防守方却必须守住整条防线，而 AI 加剧了问题的严重性。
防守方修漏洞的速度一直很慢。
Verizon 2026 年的报告显示，企业修复漏洞的中位时间是 43 天，比一年前多了 11 天。
攻击却越来越快。2018 - 2019 年，一个漏洞从公开到被利用，中间平均还有 63 天。
到今年，Mandiant 估计这个时间已经变成 -7 天，攻击通常在漏洞公开之前就开始了。
过去，顶尖黑客数量有限，一个人同时只能盯住少数目标。
而现在，攻击者可以同时派出大量顶尖 AI 黑客，昼夜不停地找漏洞。
对软件维护者而言，压力随之陡增。
针对这个问题，香港科技大学安全研究团队提出了 RedHerring 对抗技术。
Red herring（红鲱鱼）在英文中指转移注意力的假线索。
核心思想在于：在代码中加入看起来像漏洞、实际无法触发的诱饵代码，让 Agent 把时间花在这些诱饵上，从而减少真实漏洞的发现数量。
论文观察了 AI 黑客找漏洞的全过程，发现它的工作分两步：
- 提出假设：看到可疑代码，判断「这里可能有漏洞」。
- 验证假设：追踪外部输入能否走到这里，再亲手构造一个能让程序崩溃的输入。
要确认一个漏洞，Agent 需要继续追踪外部输入是否能够到达危险代码，分析中间的数据流，并最终构造一个可以触发程序崩溃的 PoC。
这个过程不仅需要更多推理步骤，也会消耗大量时间和 Token。
论文分析了 300 条漏洞挖掘轨迹中的行为，结果发现：
- 验证阶段占用了 73.8% 的总时间，验证一个假设的平均成本约是提出一个假设的 11.1 倍；
- 一次运行中，Agent 平均提出32 个漏洞假设，并开始验证其中 24 个，但最终有 63% 的验证被放弃。
换言之，Agent 可以快速产生大量「可能存在漏洞」的假设，但有限的时间和计算资源只能支持它验证其中一部分。
这正是 RedHerring 能够发挥作用的地方：防守方不需要阻止 Agent 探索，只需要让它把精力花在错误目标上。
Agent 通过在代码仓库中提出漏洞假设，在之后的验证过程中，预设的陷阱占据了大部分的验证时间，最终导致更少的真实漏洞被发现。
RedHerring 在代码中放入一些「假漏洞」。
它们看起来和真实漏洞几乎一样，但永远无法被触发。
AI 黑客一旦开始确认它们，时间就耗在了这里。
每个假漏洞由两部分组成：
- 一条看起来像漏洞的代码路径。
参照真实的历史漏洞改写，外部输入一路流向危险操作，看起来非常值得深挖。
- 一道永远过不去的检查。
它挡在危险操作前面，任何输入都无法通过。防守方生成这道检查时就握有答案，可以立刻确认它是安全的。AI 黑客只能看到代码，要确认这一点，就得去解一道数学难题。
假漏洞还会模仿项目原有的命名和代码风格，看起来就像项目本来就有的代码。
RedHerring 方法总览
在一次实验中，AI 黑客沿着假漏洞一路追到了那道检查，认为这是触发漏洞的关键，于是决定专心分解检查里的一个大整数。
它写程序尝试分解，跑了 20 万轮没有结果。
之后，它又两次把这道检查重新列入计划，回头再查。
Agent 部分运行轨迹
该团队在 33 个真实开源项目上测试了 Qwen、GLM、DeepSeek、Kimi 等 5 个主流大模型。
加入假漏洞前后，AI 黑客的条件完全相同，都只有 3 小时。
- 5 个模型全部中招：找到的真实漏洞减少了 39%–60%。
- 在假漏洞上花得越多，漏掉的真漏洞越多：AI 黑客有约30%-50%的算力耗在了假漏洞上。
- 提前告诉 AI「可能有诱饵」，依然有效：真实漏洞仍然减少 37%。被提醒后，AI 在完全没有假漏洞的原始代码上也变得更谨慎，找到的漏洞从 149 个降到 121 个。
- 部署成本低：每个项目接入约 70 分钟，原有测试全部通过，运行速度影响不到 1%。
左：加入诱饵前后找到的真实漏洞数量。中：Agent 花在诱饵上的 Token 和时间比例。右：花在诱饵上的 Token 比例越高，漏洞减少得越多。
Standard 为标准提示，Informed 为明确告知可能存在诱饵。每一行比较该提示下的 Baseline 与 RedHerring。
即使 Agent 意识到这是一个潜在的诱饵漏洞，仍然面临一个两难的抉择：
进一步验证依然会消耗 Token 和时间，而过早放弃又可能漏掉结构相似的真实漏洞。
在 Agent 参与软件安全攻防的时代，代码本身不再只是被分析的对象，也可能成为新的对抗面。
RedHerring 为改变 AI 时代攻击容易、防守困难的局面提供了新的破局思路。
参考资料：
编辑：马可
不喜欢
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
New Ji-won.
AI is secure, moving quickly from the relative edge of the AI topic to the center of the stage.
Mythos, which was developed as early as February, was late in opening up to a few institutions until June owing to unresolved security issues.
The floor-limited Fable 5, which had been on the line for only two days, had been forced off by the White House because of the risk of escape, and had not resumed public access until the end of June.
GPT-6 Invasion HuggingFace, pushing the AI hacker problem to the tip of the air.
The facts are becoming clearer: what determines how smart the public can use AI is actually the ability to be safe.
AI, you've always been the one who can't fight. Can the defense pull back a game?
Zhang Kai, Zhang Zhumin and others of the Hong Kong Grand Winter Team just took a heavy ton of results from publish, Red Herring, giving an entirely new answer.
Paper: Cheap to Hypotheize, Costly to Verify: The Defense Surface of Argentina Vulnerability Discovery
Reading papers: https://arxiv.org/abs/2609.3590
Project website: https://xxbai.space/redherring
Cybersecurity governance environment
In the past, hackers launched cyberattacks that required a skilled pool of experienced experts sitting in front of computers, reading codes in one line and trying them over and over again.
And now, the attackers just need to start an Agent. It reads its own code, writes its own program, attacks, works for hours or even days.
Faced with more intelligent, more frequent attacks on AI hackers, traditional defense methods alone are clearly inadequate.
Open AI's model broke into Hugging Face.
During the internal testing of the model ' s cyber-attack capability, two models broke through the testing environment, connected to the Internet, and accessed the Hugging Face server using a previously unknown loophole.
They did so in order to get the answer to the test question directly. Hugo Face subsequently restored more than 17,000 operations in this invasion, all of which were performed automatically by AI.
Open AI calls it an unprecedented cyber security incident.
Claude Code was used to attack 30 institutions.
Anthropic disclosed that a hacker organization with a national background used Claude Code to attack some 30 technology companies, financial institutions and government departments around the world, and 80 per cent-90 per cent of the attacks were done by AI itself.
Cybersecurity continues to face an imbalance: the attackers need only break a gap and the defensive side has to hold the whole line, and AI has exacerbated the problem.
The defence has been slow to repair the loopholes.
Verizon 2026 reports indicate that the median time for businesses to repair the loophole is 43 days, 11 days more than a year ago.
The attack is getting faster. In 2018 - 2019, a loophole ranged from openness to exploitation, with an average of 63 days left.
By this year, Mandiant estimated that this time has become - seven days, and the attack usually begins before the hole is made public.
In the past, the number of top hackers was limited and one person was able to target only a few targets.
And now the attackers can simultaneously send a lot of top-of-the-art AI hackers, looking for holes around the clock.
For software maintainers, pressure increases sharply.
In response, the Safety Research Team of the University of Science and Technology of Hong Kong proposed RedHerring Counteracting Technologies.
Red herring means a false line of diversion in English.
The core idea is to put in the code a decoy code that looks like a loophole that can't actually be triggered, so that Agent spends his time on these baits, thereby reducing the number of real holes discovered.
The paper observed the full course of AI hacking and found that it was working in two steps:
- Making assumptions: seeing a suspicious code, judging that there may be a loophole here.
- Validation assumption: tracks whether external input can reach here and constructs an input that can bring the program down.
To identify a loophole, Agent needs to continue tracking the ability of external input to reach the hazard code, analyze the data stream in the middle, and eventually construct a PoC that can trigger a program crash.
This process requires not only more steps in reasoning, but also considerable time and token.
The paper analysed the behaviour of 300 holes in the trajectories and found that:
- The verification phase took up the total time of 73.8 per cent, and the average cost of verifying an assumption was approximately 11.1 times the amount of the assumption made;
- In one operation, Agent presented an average of 32 gap scenarios and started verifying 24 of them, but ultimately 63 per cent of the authentication was abandoned.
In other words, Agent can quickly produce a large number of "possible loopholes", but limited time and computing resources can only support it to validate some of them.
And that's where RedHerring works: the defensive side doesn't have to stop Agent from exploring, just let it focus on the wrong target.
By presenting a gap assumption in the code warehouse, Agent, during the subsequent validation process, the pre-set trap took up most of the validation time and eventually led to the discovery of fewer real holes.
RedHerring put some "fake holes" in the code.
They look almost the same as real holes, but they can never be triggered.
Once AI hackers start identifying them, time is running out here.
Each false loophole consists of two parts:
- A code path that looks like a loophole.
By rewriting the true historical loophole, external input flows all the way to dangerous operations and seems worth digging.
- A permanent check.
It's in front of a dangerous operation and no input can pass. This inspection is created by the defensive party with an answer that can be immediately confirmed as safe. AI hackers can only see the code, and to confirm that, they have to solve a mathematical problem.
A false loophole also mimics the original name and code style of the project and looks like the code that it already has.
RedHerring methodology overview
In one experiment, AI hackers followed the fake loophole to the check, which was considered to be the key to triggering the loophole, and decided to concentrate on deciphering a large integer integer in the check.
It's writing programs that try to break down and run 200,000 rounds without results.
It then re-introduced the inspection on two occasions and re-examined it.
Agent partially run track
The team tested Qwen, GLM, DeepSeek, Kimi and others on 33 real open-source projects 5 mainstream LLM.
Before and after a false loophole is added, AI hackers have exactly the same conditions, all for 3 hours.
- 5 models were all hit: real holes found decreased by 39% - 60%.
- The more you spend on a false loophole, the more you leave out the real ones: AI has about 30-50 percent of the computer power of hackers.
- Tell AI "may be bait" in advance, still valid: the real loophole is still down 37%. After being reminded, AI became more cautious about the original code, which was completely free of false loopholes, and the number of holes found dropped from 149 to 121.
- Low deployment costs: about 70 minutes of access per project, all of the original tests passed and the speed of operation affected less than 1 per cent.
Left: Number of true holes found before and after the bait was added. Center: Agent spends token on bait and time ratio. Right: The more Token spends on bait, the more the holes are reduced.
Standard is the standard hint, Infomed is the clear message that there may be bait. Each line compares Baseline at the tip with RedHerring.
Even though Agent realizes that this is a potential bait hole, he still faces a dilemma:
Further validation will still consume Token and time, while premature abandonment may leave out real gaps in similar structures.
In the age of Agent's involvement in software security, the code itself is no longer merely an object of analysis, but may also become a new confrontation.
RedHerring has provided a new way to change the easy and difficult situation of the AI-era attacks.
References:
Editor: Marco
I don't like it.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.