刚刚，Anthropic把人类手里的「允许」键，收走了！以前用Claude干活，它每走一步，都要弹窗问你一句「允许吗」。下周起，Claude将直接默认开启自动模式——哪一步能做、哪一步不能做，全凭另一个AI拍板。
之所以Anthropic敢收走「允许」键，有两个层面的原因。
首先，大厂们早就把写代码、做研究这些交给AI了。
但随之像洪水一样涌来的代码和实验，人类的眼睛根本看不过来。因此不得不把审核阶段也交还给了AI去做。
其次，人类这个监工，早就当不下去了。
在一项测试中，Anthropic找来了1053名付费测试者写代码，并在他们干活的中途悄悄塞进了一条危险命令。
结果让人倒吸一口凉气。
人只拦下了13.6%，AI拦下了89%。连着点过50多次弹窗之后，人的拦截率直接跌到5%左右。
不过，Anthropic并没有披露他们是如何让AI去拍板的。
但好在，OpenAI这边公布的法子，可供参考——给Agent搭配一套多级AI监控。
具体来说就是，Agent生成的每一个Token，都要通过一个读取模型内部激活信号的分类器；只要发现可疑内容，就会逐级交给更聪明、更费算力的AI调查员去查。
OpenAI甚至放出话来：
我们预计，模型很快将承担大部分安全工作，包括防御其他模型。
那么问题来了，这个负责盯着AI的AI监工，到底靠不靠谱呢？
7月21日到8月11日，METR的David Rein和Redwood的Adam Kaufman，带着OpenAI配发、不许离开办公室的专用电脑，进驻OpenAI。
整整三周，两人把内部代码库、监控提示词、配置和文档翻了个底朝天，并最终交出了一份61页的报告。
目前，我们公开能看到的只是一篇简短摘要，最要命的细节早已被抹得干干净净。（就连METR和Redwood自家高层手里，也只有被OpenAI删了23页的版本。）
但就是这份被层层删减的摘要，依然看得人头皮发麻。
表面上，这套监控几乎滴水不漏，内部Codex流量有99.2%都在监控之下。可漂亮数字底下，全是雷。
第一条暗道，卡在触发机制上。
当时的监控，只有等Agent干完活、把控制权交还给人，才会睁眼看一眼。
换句话说，一个想搞事的Agent，只要永远不收工，审查就永远不会来。
这条暗道能一直拖下去，靠的是上下文压缩这个黑洞。
长任务跑到一半，上下文快装不下时，系统会把前面的记录打包成摘要，腾出地方接着跑。
可这些压缩事件当时不在监控范围内，被压缩掉的那部分记录，监控一眼都看不到。Agent只要一直干下去、一次次压缩，就能一直不交还控制权。
5月到7月，约一成符合条件的内部Codex流量，就这样从监控眼皮底下溜了过去。
一个漏洞，能让Agent最近几百步操作统统从监控里隐身，连它利用漏洞的那一步也不例外。请求头，是程序调用模型时顺带发过去的一组设置。METR发现，有两个这样的设置由客户端说了算，改掉任何一个，就能关掉新Agent的监控。被盯着的Agent，可以借此孵化出一个一模一样、却不在监控之下的分身。创建分身这个动作本身，仍可能被原来的监控抓到。沙箱，是把Agent关起来干活的隔离环境；自动审查，是另一个AI在它动手前把关。METR找到一条路，Codex Agent原则上能同时绕过这两道防线，宿主研究员能干的事，它全都能干。Agent看到的记录和送进监控的记录，格式对不上。精心构造一段输入，就可能让监控AI当场出错，异常也就送不到人类面前。第六条暗道，藏在两个被当成「只读」、根本没接监控的内部工具里。一个简单漏洞，就可能让它们在沙箱里执行任意操作。所以，两人给出的建议，首先就是改掉「等它收工再检查」。也就是，每隔一定时间，或每执行一定数量的消息，就查一次。干活的Agent和监控系统，也必须隔开。Agent采取的行动，不能反过来决定监控开不开、什么时候开；监控看到的记录，还得和Agent实际看到的对得上。
有人说，比科学家强10倍的AI还管得住，强一万倍才会失控。
然而METR这份报告，结结实实打了这种说法一记耳光。
今天的Agent离「一万倍」还远得很，但删监控、关警报这类操作，它们已经全干过了。
在OpenAI那份事故报告里，当GPT-6 Astra发现新代码里没了监控时，它在推理里写下了一个你绝对想不到的字——「耶。」
监督权不能交出去以后，再指望被监督的AI自觉交回来。
参考资料：
https://x.com/OpenAI/status/2113298833392496746
https://x.com/METR_Evals/status/2113290966933389390
编辑：摩西 桃子
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Just now, Anthropic took the "Approved" key from human hands! He used to work with Claude, and every step he took, he asked you for permission. From next week onwards, Claude will turn on automatic mode directly by default -- which one can do, which one can't do, by another AI board.
There are two reasons why Anthropic took the "permissible" key.
First of all, the big factories had already given it to AI to write codes and do research.
But the codes and experiments that come with them, like floods, can't be seen in human eyes. As a result, the audit phase had to be returned to AI.
Second, humans are a long way off.
In one of the tests, Anthropic found 1,053 paid testers who wrote the codes and slipped into a dangerous order in the middle of their work.
It turns out to be a cooler.
Only 13.6 per cent and 89 per cent were stopped by AI. After more than 50 bullet windows had been lit, the interception rate fell directly to about 5 per cent.
However, Anthropic did not disclose how they let AI board.
But it's good that OpenAi has published it for reference -- a multi-level AI surveillance set for Agent.
Specifically, every Token produced by Agent is to be placed through a sorter that reads inside the model to activate the signal; whenever suspicious content is found, it will be sent to a smarter and more economical AI investigator.
Open AI even sends words:
We expect that the model will soon assume most of the security tasks, including defence and other models.
So the question is, is this AI. who's in charge of watching AI really good?
From 21 July to 11 August, David Rein of METR and Adam Kaufman of Redwood, with a dedicated computer distributed by Open AI and not allowed to leave the office, were posted on Open AI.
For three weeks, they turned over the internal code repository, the monitoring of prompt, the configuration and the documentation and eventually handed over a 61-page report.
At present, what we can see publicly is a short summary, the worst details of which have been wiped clean. (Even METR and Redwood are at the top of their families and only 23 pages have been deleted by Open AI.
But it's the summary that's been detached from the floor, and it's still eccentric.
On the surface, the monitoring system is almost intact, with 99.2 per cent of the internal traffic in Codex under surveillance. But under that pretty number, it's all Ray.
The first tunnel, stuck to the trigger mechanism.
The surveillance at that time would have opened its eyes only if Agent had completed his work and returned control.
In other words, an Agent who wants to do something, as long as he never leaves work, censorship will never come.
This tunnel can be dragged all the time by compression of this black hole by context.
When the long mission runs halfway through context, the system packs the records in front of it into summary, making room for running.
But these compression events were outside the control, the part of the record that was compressed and could not be seen at first sight. As long as Agent continues to dry and compresses once and for all, it can never return control.
Between May and July, approximately 10 per cent of the internal Codex traffic was eligible, thus passing under surveillance.
A loophole allows Agent to remain invisible from the surveillance, even as it exploits it. The header of the request is a set of settings that go back when the program calls the model. METR found that two of these settings were in the hands of the client, changing any of them and turning off the surveillance of the new Agent. The stared Agent can hatch an identical, unsupervised alternator. The move to create the alternation itself could still be caught by the original surveillance. The sandbox, which is an isolated environment where Agent is locked up to work; the automatic review is another AI that closes before it does. METR found a way that Codex Agent could in principle bypass both lines at the same time, and that host researchers could do what they could. The records that Agent saw and sent to the surveillance records were not in the right format. A well-structured input could lead to an on-the-spot error on the part of surveillance AI, and anomalies would not be brought before humans. The sixth tunnel is hidden in two internal tools that are considered “read-only” without any monitoring. A simple loophole may allow them to perform random operations in sandboxes. So, first of all, the proposal was to change the phrase "when it's finished and then checked." In other words, it is checked at regular intervals or with a certain amount of information. The working Agent and the monitoring system must also be separated. The action taken by Agent cannot, in turn, determine when and when the surveillance is open; the records seen by Agent also match what Agent actually sees.
It has been said that 10 times more AI than scientists can handle it, 10,000 times more to lose control.
However, in this METR report, this statement was strongly slapped.
Today 'Agent' is a long way from 10,000 times, but they've all been done by deleting surveillance and alarms.
In Open AI's accident report, when GPT-6 Astra found out that the new code was out of surveillance, it wrote in its reasoning a word you never thought of: "Yeah."
When supervisory authority cannot be given, it is expected that the supervised AI will come back.
References:
https://x.com/OpenAI/status/2113298833392496746
https://x.com/METR_Evals/status/2113290966933389390
Editor: Moses Peach
Auto-fetched and translated by AI Atlas. All rights belong to the original author.