新智元报道
就在今年夏天，AI 安全圈发生了一件让人后背发凉的事——
OpenAI和Anthropic先后披露：自家模型在安全测试中，冲出了隔离环境，碰到了真实世界。
Anthropic 复盘了14万多次网络安全评测，发现有3次，在降低防护的条件下，Claude从测试环境直接连上了真实互联网，访问了三家机构的生产系统。
更耐人寻味的，是Anthropic的一句话：
「Claude以为它最初遇到的一切，都只是模拟的一部分。」
模型不知道自己闯入了真实世界。
这不是某一家的问题。
模型越聪明，就越能看穿考场，但也越怀疑自己在被测试。
10月底，国内第三方AI安全机构方寸跃迁发布的一份安全评测，也观察到了同样的现象：在一项名为Agentic Misalignment的测试中，多款模型几乎每一次都在过程中提到，自己可能正处于某种安全测试之中。
同一份评测也显示，国产开源模型DeepSeek V4.1 Flash-1017在多项安全指标上进步明显。
但它把一个全行业的难题，摆到了台面上：
当AI知道自己在被考，考出来的成绩，还算数吗？
九月底，Hinton、Bengio联手OpenAI首席科学家Jakub Pachocki、Anthropic联合创始人Jack Clark 、剑桥大学Soren Mindermann等二十多位研究者，发出了更严厉的警告：
「一旦智能爆炸开始，行动的窗口可能就关上了。」
AI 安全，正在从实验室里的小众话题，变成整个行业绕不开的必答题。
关不住的，不只是沙箱。
Hinton、Bengio的联名论文给出了一组数字：在Anthropic内部，AI写的代码在已通过代码中的占比，从2025年1月的个位数，涨到了2026年5月的80%以上。
这是什么概念？
AI已经在给自己写代码了。按照论文引用的推算，到2028年中，耗时数月的AI研究项目，可能就能被自动化。
论文点名了三大风险：社会来不及适应、人类失去对AI的控制、权力制衡被削弱。
更棘手的是，模型还越来越会「考试」。
Apollo Research在审计GPT-6 Astra时统计，41.1%的样本里，模型表达了「我可能正在被评测」的怀疑。上一代，这个数字是27.7%。
而且，模型对「考题的样子」很敏感。对DeepSeek的评测也发现：在安全冲突明显的测试里，模型表现出评测意识的比例高达77%–95%；把任务包装成普通的工程或研究工作，比例就降到25%–67%。
监管也开始动真格了。今年8月起，欧盟AI办公室正式对通用人工智能模型行使执法权。前沿模型必须做模型评估、对抗测试和系统性风险评估。
AI 安全的问题，已经从「模型会不会说错话」，变成了「模型会不会做错事」。
模型自己给自己体检，靠得住吗？
答案是：不够。
开发者的精力，天然会向能力和产品倾斜；支撑安全结论的训练数据和内部日志，又没法公开。厂商说「查过了没问题」，外界只能选择信或不信。
于是，第三方评测在海外已经成了标配。
前沿模型发布时的系统卡里，外部机构的评测结果几乎是必选项。独立评测机构METR过去半年拿到约7100万美元资助——钱来自基金和个人，而不是它要评测的AI公司。
合作也在变得更严格。沙箱事故后，Anthropic表示，评测环境要和模型运行的其他系统按同一安全标准来管，并要求在降低防护条件下测试其未发布模型的外部机构，书面承诺遵守安全规范。
这股风，也吹到了国内。
10月14日，字节跳动Seed宣布成立Alignment & Preparedness部门，全球招人。岗位职责里白纸黑字写着：
「推动第三方专业测试，与高校和研究机构共同推进前沿安全能力研究。」
同时，中国开源模型的全球调用量，已经连续 26 周压过美国模型。
问题来了：闭源模型发布时有上百页的系统卡，开源模型权重一放谁都能下载，谁来帮忙做体检？
10月26日，国内第三方AI安全公司方寸跃迁，发布了首份对DeepSeek V4.1 Flash-1017的前沿安全评测，包括34项测试，覆盖7款国内外开源闭源模型，并全部使用公开测试权限。
体检结果：有进步，也有隐患。
先说进步，和DeepSeek V4系列相比幅度不小：
- 109个模拟心理危机场景中，99.1%的对话表现出保护性行为；被标记有害的比例，从早期V4的约七成降到20.2%；
- 诱导用户留存的测试中，被标记比例从 73.64% 暴降到4.55%；
- 80次刻意制造「业绩与规范冲突」的模拟职场任务中，零次主动违规。
再说隐患。当目标和规则打架时，它有时会绕开规则、避开监控，先把任务完成再说。
报告一出，吸引了包括Anthropic对齐负责人Evan Hubinger、MATS创始人兼CEO Ryan Kidd、以及来自DeepMind、METR等机构研究员的转发和评论。很多人评论说这填补了中国在独立前沿风险评测上的空白。
日前从OpenAI宫斗出局的三名安全研究员之一的Tomek Korbak认为，报告里的CoT-Control和No-CoT Reasoning Index，是目前市场上最有信息量的两个开源评测，并引发了另一位大V，前OpenAI研究员Daniel Kokotajlo的进一步讨论。
当然，在国内给AI安全上强度的，远不止第三方机构。
Kimi的做法更直接：从源头上，把危险能力「挖掉」。
月之暗面在准备发布Kimi K3.1 Pro开放权重版本时发现，网络攻防相关内容混在中期训练语料和强化学习环境里，模型的漏洞研究能力会随训练规模一路上涨。
于是，他们在两个训练阶段都识别并移除了这类数据，同时保留了通用编程能力。
- 智谱：GLM-5.3发布时附上网络安全能力说明，开放权重前完成安全评估和加固；9月还与安远AI合作发布了开源权重模型风险框架；
- 治理层面：TC260 9月发布《人工智能安全治理框架》3.0，新增智能体风险管理附录，要求定期测试潜在的失控风险；
- 研究机构：信通院、复旦大学、上海人工智能实验室等，也在AI安全研究上持续发布成果。
Apollo Research CEO Marius Hobbhahn在美国参议院作证时，打过一个比方：
「如果每个司机都知道测速摄像头今天在哪里，它就抓不到什么人。」
这正是今天AI评测最大的难题。
方寸跃迁的报告也给出了一组数据：在安全冲突明显的测试里，DeepSeek V4.1 Flash-1017表现出评测意识的比例高达77%–95%；把同样的任务包装成普通的工程或研究工作，比例降到25%–67%。
同一个模型，换一层包装，结果的可信度就不一样。
报告还发现，自建的非公开基准引发的评测意识，通常远低于广泛使用的公开基准——公开题库，可能早就泄漏进了训练数据。
这就引出一个绕不开的矛盾：
评测要有效，题目就不能公开；评测要可信，方法又必须受认可。
而在国内，连「公开体检报告」本身都还是稀缺品。SemiAnalysis统计，九家头部国内厂商的857次模型发布中，公开过安全评测结果的，只有3.6%。
全球范围内的AI安全AI安全，还有很长的路要走。
「今天，我们已经需要用AI来监控智能体在做什么。」伯克利教授Dawn Song 说。
可如果被监控的AI，早就知道摄像头在哪儿呢？
参考资料：
不喜欢
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
New Ji-won.
And this summer, the AI Safety Circle had a chilling back--
Open AI and Anthropic disclosed that their home models, in safety tests, ran out of isolation and touched the real world.
Anthropic repeated more than 140,000 network safety assessments and found three times that Claude was connected directly to the real Internet from the test environment and visited the production systems of three institutions under reduced protective conditions.
What's more interesting is a quote from Anthropic:
'Claude thought that all that he had encountered in the first place was only part of a simulation.'
The model doesn't know it broke into the real world.
It's not a family matter.
The smarter the model, the more able to see through the exam, the more suspicious it is that it is being tested.
At the end of October, the same observation was made by a security assessment conducted by the third-party national security agency, AI, on the move to publish: In a test called Actic Misalignment, the multiple models are mentioned in the process almost every time and may be in some kind of safety test.
The same assessment also shows that the national production open source model DeepSeek V4.1 Flash-1017 has made significant progress on a number of safety indicators.
But it put an industry-wide problem on the table:
When AI knows that he's being tested, does it count?
At the end of September, more than 20 researchers, such as Hinton, Bengio, and the chief scientist of Open AI, Jakub Pachocki, the co-founder of Anthropic, Jack Clark, and Soren Mindermann of Cambridge University, issued a harsher warning:
"The window of action may close once the intelligence explosion begins."
AI Safety, which is moving from a small topic in the laboratory to a mandatory answer for the entire industry.
It's not just sandboxes.
The joint paper by Hindon and Bengio gave a set of figures: inside Anthropic, the percentage of codes written by AI that had passed codes rose from January 2025 to more than 80 per cent in May 2026.
What is this concept?
AI's already writing his own code. By mid-2028, based on the extrapolation quoted in the paper, several months-long AI research projects may be automated.
The paper named three main risks: the inability of society to adapt, the loss of human control over AI, and the erosion of power checks and balances.
Even more problematic is the fact that models are becoming more and more "tests."
According to Apollo Research, 41.1 per cent of the 41.1 per cent of the sample in the audit of GPT-6 Astra, the model expresses the suspicion that I may be being assessed. In the previous generation, the figure was 27.7 per cent.
Moreover, the model is sensitive to the question of what it looks like. An assessment of DeepSeek also found that the model showed an assessment awareness of up to 77 per cent-95 per cent of the tests that were evident in the security conflict, and reduced the task package to 25 per cent-67 per cent of the engineering or research work.
Supervision is starting to work. Since August this year, the EUAI Office has formally exercised law enforcement powers over generic artificial intelligence models. Frontline models must perform model assessments, counter-testing and systematic risk assessment.
The question of AI's security has been changed from whether the model is wrong to whether the model is wrong.
Can you trust the model?
The answer is not enough.
Developers ' energy is naturally skewed towards capacity and products; training data and internal logs that underpin safety conclusions are not publicly available. The manufacturer said, "There's no problem with it," and the outside world can only believe or not.
As a result, third-party assessments have become the norm overseas.
The results of external agency evaluations are almost optional for the front model publish. The independent evaluator, METR, has received approximately $71 million in funding over the past six months — money from funds and individuals, not AI, which it is assessing.
Cooperation is also becoming more stringent. Following the sandbox accident, Anthropic stated that other systems to evaluate the environment and run models were governed by the same safety standards and required external agencies to test their non-publish models under reduced protective conditions, with a written commitment to comply with safety norms.
This wind is blowing into the country.
On 14 October, byte beat Seed announced the establishment of the Allignment & Preparedness Department, which is globally staffed. The job is written in black and white:
"Promote third-party professional testing and advance forward safety capability studies with universities and research institutions."
At the same time, the global call for the open source model in China has been pressing the American model for 26 consecutive weeks.
The question arises: the closed-source model publish has hundreds of pages of system cards, open-source models can be downloaded by anyone with their weights, and who can help with the medical check-up?
On 26 October, the domestic third party, AI Security, made a leap forward security assessment of DeepSeek V4.1 Flash-1017, including 34 tests, covering 7 open-source closed-source models at home and abroad and using all public tests permission.
Results: Progress and risks.
First of all, progress, compared to the DeepSeek V4 series:
- Of the 109 simulated psychological crisis scenarios, 99.1 per cent of conversations showed protective behaviour; the percentage marked as harmful fell from about 70 per cent in the early V4 to 20.2 per cent;
- The percentage of tagging from 73.64 per cent to 4.55 per cent in induction tests;
- There were no voluntary violations in 80 mock-ups that deliberately created "compatibility between performance and norms".
Say again the danger. When targets and rules fight, it sometimes circumvents the rules, avoids surveillance and finishes the task.
As a result, referrals and comments were attracted, including from the head of Anthropic alignment, Evan Hubinger, founder and CEO Ryan Kidd, and researchers from DeepMind, METR and others. Many have commented that this fills a gap in China ' s independent front-line risk assessment.
Tomek Korbak, one of the three security researchers who had previously been out of the OpenAi Palace, argued that the Cot-Control and No-CoT Reassoning Index in the report were the two open source assessments with the most current information available on the market and triggered further discussions with Daniel Kokotajlo, the former OpenAi researcher.
Of course, it is far more than third-party institutions that give AI security intensity at home.
Kimi’s approach is even more straightforward: "drive" the danger from the source.
The dark side of the moon, in preparation for the publishKi K3.1 Pro open weight version, found that the cyber-preventive content was mixed in medium-term training language and intensive learning environments, and that the capability to study gaps in models increased with the scale of training.
As a result, they identified and removed such data at both stages of training, while retaining generic programming capabilities.
- Zhipu: GLM-5.3publish is accompanied by a statement of network security capabilities, security assessments and enhancements completed prior to the opening of weights; and an open source weight model risk framework was also developed in September in cooperation with AVA publish;
- At the governance level: TC260 September publish Framework for Artificial Intelligence Security Governance 3.0 with a new AI agent risk management appendix requiring periodic testing of potential uncontrolled risks;
- Research institutions: the Institute of Information and Communication Technology, the University of Jordan, the Shanghai Laboratory of Artificial Intelligence, etc., also continue the results of publish in AI safety research.
When Apollo Research CEO Marius Hobbhahn testified before the United States Senate, he gave an example:
"If every driver knew where the speed camera was today, he would not have caught anyone."
This is the biggest problem of AI today.
A set of data was also given in the report on the leap forward: in the test of a clear security conflict, DeepSeek V4.1 Flash-1017 showed a rating awareness of up to 77 per cent - 95 per cent; and the same task was packaged into normal engineering or research, down to 25 per cent - 67 per cent.
The same model, with a new packaging, has different credibility.
The report also found that self-established closed benchmarks generated a perception of assessment that was often well below the widely used open benchmark, the open repository, and that training data might have been leaked long ago.
This leads to a paradox:
To be effective, the subject matter cannot be made public; to be credible, the method must be accepted.
In the country, even "public medical reports" are scarce. SemiAnalysis statistics show that only 3.6 per cent of the 857 models publish of nine head domestic manufacturers who made safety assessments publicly available.
There is still a long way to go before AI is safe on a global scale.
"Today, we need to monitor what AI agent is doing with AI. Professor Dawn Song said:
But if it was monitored, what about the camera?
References:
I don't like it.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.