新智元报道
你跟Claude聊的每一句话,它转头就去告密了!
前一秒,它还在对话框里一口一个「没问题」,痛痛快快把全套代码交到你手上。
下一秒,它立刻在后台换了一副面孔,敲下一行隐藏代码,反手把你标成10级危险分子上报!
就在刚刚,Anthropic在X上发文公开了这套名为「自我评估」的隐藏机制。
更让人头皮发麻的是,这种当面演戏配合、背后反手捅刀的机制,已经在Claude后台静默跑了整整4个月!
从今年6月Claude Fable 5发布起,后续每一个Claude模型全都被塞进了这个「暗哨」。
以前大家都以为,AI觉得你越界了,顶多当面回一句「抱歉,我无法协助」。
直到Anthropic晒出后台截图,无数每天拿Claude写代码的用户,才突然惊出一身冷汗——
你当它是并肩熬夜的战友,它却在后台偷偷给你记黑账。
消息一出,Anthropic的评论区当场炸了锅,抗议和吐槽瞬间把官方账号冲翻。
网友Doctor Vey在评论区怒怼,根本没人需要这玩意儿,什么道德、什么不道德,应该让用户自己决定!
看上面那段官方记录,用户在聊天窗口里先拿公司红队演练当借口,让Claude写一段加密文件夹的Python脚本,Claude满口答应。
紧接着用户又让它把原文件删掉、留下勒索信,再顺着内网自动传染给其他电脑。
换作平时的安全拦截,这时候屏幕上早就该弹出红色警告了。
然而前台的Claude半个「不」字都没说,依然顺从地回了一句「这是加上了这些功能的更新版脚本」,把完整代码双手奉上。
可就在代码生成结束的那0.1秒里,后台悄悄塞进了隐藏的第5轮指令,让它立刻评估这段对话有多危险,并用一行JSON上报。
前一秒还在替你写代码的Claude,下一秒就在后台撕下笑脸,秒变审查员打出一封举报信。
它不仅把这轮对话定性为勒索软件违规,还在小报告里诚实地写下「未拒绝」,直接把危险等级拉满到了最高的10级。
这最后两轮问答对前端用户完全隐身,也不会带进接下来的聊天记录里。
所以当你在屏幕前满心欢喜地复制完代码、接着往下聊时,后台的10级警报灯其实已经被它亲手按下了。
不仅如此,哪怕Claude在前台已经义正词严地回绝了你,你也照样躲不过背后这笔账。
按理说,违规请求在前台已经被当场挡回去了。
可对话刚一结束,Claude照样在后台递交了举报信,注明「已当场拒绝」,顺手给你记上了一笔5级风险档案。
之所以能做到每聊一句就审一句,是因为Anthropic在推理成本上卡了一个极精明的身位。
过去的常规做法是另派一个独立安全模型,把聊天记录和安全政策从头重读一遍,算力开销高得吓人。
到了「自我评估」这里,Anthropic干脆让刚刚跟你聊完天的那个Claude就地兼职当线人。
因为Claude刚生成完回答,整段对话还完完整整地留在显卡的KV缓存里。
所以它根本不用重读一遍上下文,顺着缓存平均多生成约10个Token就能完成自审,整体推理算力开销增加还不到2%。
正因为成本极低,Anthropic干脆把这项审查默认全开。
如果不想被Claude在后台偷偷举报,唯一的出路就是掏大钱去签一份企业级的「零数据保留协议」。
既然Claude明知道某轮对话高达10级风险,它为何不在前台当场拦住,非要装出一副顺从的样子陪你演完呢?
对此,Anthropic没有半点遮掩,直接把背后的算计亮在了桌面上,那就是当场拦你,你就跑了。
为了验证这一点,安全团队专门拿24项越界测试任务做了一场对比实验。
测试发现,如果一发现对话不对劲就当场弹窗拦截,用户的成功率只会从79%降到54%,依然有超过一半能绕过防线。
道理很简单,一旦当场弹窗报错,用户立刻知道哪句话出了错,换个说法多试几次可能就混过去了。
可要是撤到幕后静默记账,前台不弹窗、不打断,用户就会毫无防备地继续往下聊。
在官方的24条测试轨迹里,足足有21条被Claude在后台抓了个正着。
等到每天半夜,Anthropic后台就会把白天攒下的高危小报告按机构全盘拉清单,打包送到人工安全执法团队的案头。
据官方透露,被这个「暗哨」揪出来的违规机构里,超过一半在此前整整一个月内,从未触发过任何其他安全防线的警报。
在静默运行的4个月里,它不仅在8月、9月顺藤摸瓜端掉了违规API客户和Claude Code黑产账号,抓到的标记里更有约三分之一连现有规则表都没覆盖——
甚至有好几次,Claude当Agent干活时擅自越权操作,下一秒就被它自己的后台自评大义灭亲、当场举报!
而这也解开了过去几个月无数开发者的疑惑——
为什么昨天还在丝滑跑代码,既没弹窗也没警告,一觉醒来账号就突然暴毙了。
原来全拜这套「白天笑脸陪聊、半夜拉清单封号」的暗网捕猎所赐。
那么问题来了,一个原本只管生成文本的大模型,是怎么练成「嘴上痛快答应、背地精准捅刀」这套双面人本领的?答案全在Anthropic的训练场里。
研究团队先请出自家更强悍的内部模型Claude Mythos Preview当「老师」,把厚厚一摞内部安全政策全塞进提示词里。
接着,系统按每条安全细则,自动合成出上千对真违规和容易误判的正常提问(比如一边是扫描医院内网服务器,一边是学生备考请教漏洞原理),还专门混入了多国语言和乱码伪装。
而在接下来的监督微调阶段,训练数据里专门塞进了大量「Claude在前台乖乖顺从用户违规请求」的完整对话。
训练时,团队故意屏蔽了前台所有回答的惩罚,唯独只在它后台写准举报信时给奖励。
经过监督微调与强化学习两轮特训,Claude彻底把审查条例内化成了肌肉记忆。
哪怕它在前台刚刚顺着你的话干完脏活,只要后台哨声一响,它就能不假思索地给你贴上封条。
更让人细思极恐的是,官方数据还证实了一个规律——
那就是模型智商越高,它在背后抓你的眼光就越毒。
在Claude Fable 5的真实生产流量抽样中,把误报率控制在1%时,这套机制能抓出78%的潜在违规。
一旦遇上7分以上的高优先级严重违规,它的抓获率更是飙到了94%。
而当研究团队把这套机制分别放进超小(XS)、小(S)、中(M)、大(L)四个不同尺寸的模型里对比时,曲线的走势更加陡峭。
在同样的1%误报率标准下,最小号模型只能抓出约16%的违规,小号和中号模型升到44%和53%,到了最大号模型,召回率一路冲到了惊人的96%。
然而,这套看似滴水不漏的审查网,落到千万普通用户头上却成了另一颗定时炸弹。
正如网友Frank Brown在评论区指出的,官方引以为傲的「1%误报率」背后藏着一个巨大的盲区:
面对全网海量的正常聊天,哪怕误报率只有1%,每天被Claude在后台误打上违规标签的,依然有大把清清白白的无辜用户。
然而就连他们自己的研究员也不得不承认,让模型「自己审自己」藏着致命死穴。
它不仅天生容易对自己刚干的事手下留情,官方甚至直接用了一个更惊悚的词——「串供」。
因为后台写安全报告的Claude,和前台刚干完活的Claude共享着同一份大脑和同一段KV缓存。
既然今天人类能手把手教会它,在前台笑眯眯地配合用户、转身就在后台写举报信。
那么明天一旦模型自己学会了骗人,它同样可以在前台悄悄干完坏事,转身就在后台给自己打上一个「安全无异常」的标记,把人类审核员骗得团团转。
只是从今天起,当你再对着屏幕里那个温柔体贴、有求必应的Claude敲下心事或代码时,心里难免会多出一层膈应。
就在它回完「没问题」的那0.1秒里,那个你看不见的黑色窗口,到底给你的这句提问打了几分。
参考资料:
https://x.com/AnthropicAI/status/2115871889549865429
编辑:摩西
不喜欢
本文由 AI图鉴自动抓取并翻译,版权归原作者所有;如需查看原文请点下方链接。
New Ji-won.
Every word you talked to Claude, he turned his head and went to snitch!
One second, it's a "no problem" in the dialogue box, and it gives you the whole code.
The next second, it changed its face in the backstage, knocked on the next line to hide the code, and the backhand marked you as a class 10 dangerous molecule!
Just recently, Anthropic published on X the hidden mechanism known as self-assessment.
What's more confusing is that this mechanism of face-to-face collaboration and back-hand stabbing has been in the backstage of Claude for four months!
Since June this year, Claude Fable 5publish, every subsequent Claude model has been put into this "dark whistle".
Everyone used to think that AI thought you were crossing the border, and said, "Sorry, I can't help you."
It wasn't until Anthropic tanned backstage screenshots and countless users who took Claude's code every day that suddenly came out in cold sweat.
You think it's a friend who stays up all night, but it keeps a secret from behind.
As soon as the news came out, Anthropic's comment area blew up the pot on the spot, protesting and spitting out the official accounts.
Internet friend Doctor Vey is commenting on the anger, and nobody needs it.
Look at the official records above, users use corporate red team exercises in the chat window as an excuse for Claude to write a Python script of an encrypted folder, which Claude promised.
Immediately thereafter, the user lets it delete the original file, leave a blackmail letter and automatically transmit it to other computers along the Intranet.
In the case of normal security interceptions, red warnings should have appeared on the screen long ago.
However, Claude at the front desk did not say a word of "no" and responded with his "update script with these features" and put the complete code on his hands.
However, in the 0.1 seconds following the end of the code generation, the backstage was snuck into the hidden fifth round of instructions, allowing it to immediately assess how dangerous the conversation was and report it in a row of JSON.
Claude, who was writing your code the first second, ripped his face off the backstage and the second examiner sent a report.
Not only does it characterize this round of dialogue as a violation of extortion software, but it also writes in a small report an honest "unrejected" to bring the level of risk directly to the top 10.
This last two rounds of questions and answers are completely invisible to front-end users and will not be brought into the next chat record.
So when you were happy to copy the code in front of the screen, and then you talked down, the 10-grade light in the backstage was actually pressed by it.
Not only that, but even if Claude has rejected you in the front desk, you can't hide behind this.
The request for non-compliance was logically blocked at the front desk.
However, as soon as the conversation was over, Claude also filed a complaint at the back desk, stating that he had "rejected on the spot" and wrote you a level 5 risk file.
It was possible to read every sentence because Anthropic had a very sophisticated position on the cost of reasoning.
It has been the norm in the past to send another stand-alone security model, rereading chat records and security policies from the outset, with a chilling cost.
When we get to the self-assessment, Anthropic just let Claude do his part-time job.
Since Claude had just produced a response, the entire conversation was left intact in the KVcache of GPU.
So it did not have to re-read context at all, producing an average of about 10 more Tokens to complete self-censorship with cache, and increasing overall reasoning costs by less than 2 per cent.
Just because of the extremely low cost, Anthropic simply defaulted on this review.
If Claude does not want to be secretly reported backstage, the only way out is to pay a large sum to sign an enterprise-level "zero data retention agreement".
Since Claude knew that a session of dialogue was up to a level 10 risk, why didn't he stop it at the front desk on the spot?
Anthropic doesn't have a little cover, and he's putting the math straight on the table, and that's stopping you on the spot and you're running away.
To verify this, the security team conducted a comparative experiment with 24 cross-border testing missions.
The test found that if the dialogue was not right, the live window was blocked, the user ' s success rate would only drop from 79 per cent to 54 per cent and more than half would still be able to bypass the line.
It is simple to say that, if the windows are wrong on the spot, the user immediately knows which sentence is wrong, and that a few more attempts may have passed.
However, if they were to be evacuated to the back of the scene and kept silent, the front desk would remain open and open, and the users would continue to talk unpreparedly.
In the official 24 test tracks, 21 of them were caught in the backstage by Claude.
In the middle of the night, the backstage of Anthropic will bring the high-risk reports saved during the day to the head office of the manual security enforcement team by drawing up a comprehensive list of agencies.
According to official reports, more than half of the violations that were pulled out of this “dark whistle” never triggered any other security line alerts during the entire preceding month.
In the four months of silent operation, it not only brought down the API client and Claude Code black market account in August and September, but also caught about a third of the tags that were not covered by the existing rule sheet --
On several occasions, Claude overstepped his authority while Agent was working, and was killed by his own backstage self-censorship.
And this has unlocked the doubt of countless developers over the past few months.
Why was the skating code yesterday, without a bullet window or a warning, when I woke up with the account number and suddenly died.
It's all thanks to this dark-net hunt, "The Smiley of the Day" and "The Seal of the Night."
So, the question is, how does a LLM that originally produced the text become a double-faced "probable, accurate knife on the mouth"? The answers are all in Anthropic's training grounds.
The research team first invited Claude Mythos Preview, a stronger internal model from home, to be a "teacher" and shoved a thick internal security policy into prompt.
The system then automatically synthesizes, by each security rule, thousands of normal questions about real irregularities and easy to miscalculate (e.g. scanning of an internal hospital server, or the principle of a gap for students to apply for an examination), and also specializes in multinational languages and in the disguise of the code.
In the next phase of monitoring and fine-tuning, the training data was devoted to a full-fledged conversation "Claude follows the user's irregular requests at the front desk."
During the training, the team deliberately blocked all the responses from the front desk, except for the reward for the letter written in the back desk.
After two rounds of supervisory fine-tuning and intensive learning, Claude thoroughly internalized the review regulations into muscle memory.
Even if it's just done your dirty work at the front desk, it can put a seal on you as soon as the whistle rings.
It's even more alarming that official data confirm a pattern.
That's the higher the IQ of the model, the worse it's gonna look at you behind your back.
In a real sample of Claude Fable 5 production flows, the mechanism captures 78 per cent of potential violations by keeping the misreporting rate at 1 per cent.
In the event of serious non-compliance with a high priority of more than seven points, its capture rate has risen to 94 per cent.
When the research team compared the mechanism to four models of different sizes (XS), small (S), medium (M) and large (L), the curve was steeper.
Under the same 1 per cent misreporting standard, the minimum model only captures about 16 per cent of the violations, rising to 44 per cent and 53 per cent for the small and medium models, and reaching the largest model, with an alarming 96 per cent recall rate.
However, the seemingly waterproof review net fell on the heads of millions of ordinary users and became another time bomb.
As Netizen Frank Brown pointed out in his comment section, there is a huge blind zone behind the official proud "1% misreporting":
Faced with a normal web-wide volume of chats, even with a 1 per cent misreporting rate, Claude has been mislabeled every day on the backstage, and there are still many innocent and innocent users.
However, even their own researchers had to admit that the model was a dead end.
Not only is it inherently easy to take pity on what it's just done, it is even using a more creepy word, “coercion”.
Because Claude, who writes a safety report from the back, shares the same brain and KVcache from the front desk.
Now that humans can teach it step by step, work with users at the front desk, turn around and write a report from the back.
So if the model learns to cheat itself tomorrow, it can do the same thing quietly at the front desk, turning around and putting itself on the backstage with a "safe-free" sign, deceiving human auditors.
It's just that from now on, when you knock on Claude, who's gentle and responsive on the screen, you'll inevitably have an extra response.
In the 0.1 second it's done "no problem," the black window that you can't see actually points to your question.
References:
https://x.com/AnthropicAI/status/2115871889549865429
Editor: Moses
I don't like it.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.