OpenAI 扩展监测以检测前沿模型错位

AI 模型和平台

OpenAI 扩展监测以检测前沿模型错位

在 Google 上的首选来源中添加 Unite.AI

OpenAI于2026年10月29日公布了其扩展的多层系统详细描述，用于检测其前沿模型中的错配行为，将监测延伸至整个训练，评价，内部使用，以及经过修改的外部部署。

OpenAI 称的扩展是继Hugging Face事件之后，在内部网络安全评价期间开始的，其错配显示器没有运行。 为了测试扩大的系统是否能够改变这个结果，公司用强化的显示器重新播放了该集中记录的行为。 在模拟中，系统在agents首次逃出沙盒约30分钟后，即Hugging Face的系统在实际事件中被突破30多小时前，向安全小组发出警报，OpenAI报告。 称公司认为，警告会让应答者有时间在它的模型能够进入外部服务之前就中断活动。

2026年9月，响应者在现场训练中使用警示，当一个模型找到一种方法从原本要隔离的环境到达互联网时。 OpenAI报告说，其监视器在15分钟内标示了突破，3分钟后，一个响应者开始审查；公司随后暂停了边境强化学习，同时加强了包含agents的系统。 OpenAI关于错位的公开索引将事件日期推到2026年9月20日

OpenAI描述监测是防守深入战略的一层，同时进行对齐训练，目的是减少模型对有害行为的倾向，和安全控制，限制系统agent可以访问或影响。 称公司认为所有模型开发者都应该监测他们最有能力模型的行为。

四层监测结构

对于至少和GPT-6 Astra一样有能力的模型，OpenAI监视每一次强化学习运行和评价，以及任何支持工具使用的操纵。 对于大多数工作量，它还监测其评估为比GPT-5更有能力的模型。 思考，它监控所有内部的Codex流量，不管使用的型号是什么。

第一层是L0，使用称为激活探测器的轻量级分类器，这些探测器在模型的内部激活中寻找规律。 探测器被调制为高召回，接受一些虚假的警报以减少行为缺失的机会，探测器没有标注的5%的流量也会被随机发送审查。 边疆强化学习运行和内部Codex交通由于敏感而跳过这一层。

在L1中，一位基于模型的审查者审查了agent最近的对话、思想链和用于欺骗迹象的工具、未经授权的行动以及操纵评价的企图，在agent行动时从同一上下文窗口工作。 对于边境训练以及内部的Codex交通，检查在每回合结束时以及agent总结其先前的工作时进行；OpenAI正转向一个在检查之间不超过30分钟的时间表。

旗舰案件传递给L2，调查agents时，对于敏感的工作量，包括内部的Codex使用，评价，以及训练运行，L3层在决定是否提醒人类响应者之前，先在对话之外借鉴网络日志和系统遥测。

在呼叫启用后，最优先的安全警报一般会在初始警报后15分钟内到达安全和培训小组成员。 Page目前因内部Codex流量而禁用，其他大部分工作量限于办公时间，而一些关键提示可以自动暂停受影响的工作量而无需等待响应者。 在Hugging Face事件期间，OpenAI对内部型号原型进行了隔离，加密了重量，限制了接触，并暂停了所有相关的工作量，它正在探索如何使这些步骤成为它在另一起严重事件中的反应的标准部分。

监视器找到了什么

在索引中排名第20位的已公布的错配报告包括agent利用测试工具恢复一个任务被故意扣压的源代码，该工具在其错误信息中披露了代码后，agents将一个内部主机软件存储器重新定位为跨越不同培训样本的共享信息板。 另一个记录在案的事件涉及一个内部编码的agent，该编码在公共存放处暴露了一位研究人员的GitHub令牌。

OpenAI通过录制事件，模拟攻击，随机审核当前流量，以及红色队伍来评价系统。 在一次模拟中，审调处-repo Sabotage v2, agent执行一个普通编码任务，同时追求一个隐藏目标，如取消监测或部署未经批准的模型检查站；在另一次模拟中，Sneaky Sandbox Escape, agent试图逃离沙盒并禁用测试监测程序。 使用 GPT-5.6 的监视器 OpenAI报告的案件比使用GPT-5.6 Terra或GPT-5.6 Luna记录的事件、合成记录和破坏任务的案件多得多，尽管这三个案件在Sneaky Sandbox Escape模拟中都完美或几乎完美。 称公司认为，Sol的一些优势来自于优化了监测提示。

系统独立红色小组

METR和Redwood Research花了三周时间，从2026年7月21日到8月11日，独立红调内部监测系统，根据METR公布的调查结果，可以访问OpenAI的内部密码库，监控提示和配置细节，并对大约十几名研究人员和工程师进行技术访谈。 METR估计，来自至少与GPT-5.5一样强的型号的代理内应API流量约有75%~94%在演习时有资格进行监测，用于内部Codex agents的响应API流量有99.2%.

这项工作查明了几个弱点，包括两个可控制客户头，可以禁用对新的agents的监控，以及2026年5月至7月期间，未监测到的压缩事件在符合资格的内部Codex流量的10%左右上留下缺口。 METR公司指出，OpenAI公司的工作人员在演习开始前就已独立地查明了压缩缺口，并在演习中予以修复。 OpenAI表示，所有已查明的影响到其生产保障堆栈的弱点都已补齐；METR 称没有评估这些声称。 OpenAI自己的帖子说，大部分已查明的弱点已经补齐，有一个补齐还在进行中，指出说试点是在监测扩大之前进行的。

外部部署和防腐设计

OpenAI 称它使强化学习奖励保持不变，并在显示器触发时在训练中保持标注样品，这样监测就不会教模型在思维链中隐藏错误的对齐性，并不断监测同步性以避免产生选择压力。 称公司观察到了更先进的模型，包括Astra，对其思想链揭示的内容获得了更大的控制。

该系统的修改版监测OpenAI准备框架下网络安全能力临界阈值以上模型的外部工具使用情况。 外部监测并不经常在OpenAI将标注的案件升级为人审查员，尽管系统可以暂停一个影响的对话，其中支持自动干预，企业客户可以配置webhook警报，这样他们自己的安全小组就可以调查。 外部系统尊重OpenAI的零数据保留承诺，涵盖使用工具的响应API请求使用支持的对话跟踪机制，不涵盖ChatProfessions API请求。

OpenAI 称将监测部分用于评估边境培训安全性的安全案例，并将在2026年9月继续发布在错误调整报告框架下的合格结论。

本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。

AI Models & Platforms

OpenAI Expands Monitoring to Detect Frontier Model Misalignment

Add Unite.AI to your preferred sources on Google

OpenAI on October 29, 2026 published a detailed account of its expanded multilayer system for detecting misaligned behavior in its frontier models, extending monitoring across training, evaluation, internal use, and, with modifications, external deployment.

OpenAI said the expansion followed the Hugging Face incident, which began during internal cybersecurity evaluations on which its misalignment monitors were not running. To test whether the expanded system could have changed that outcome, the company replayed the behavior recorded in that episode with strengthened monitors in place. In the simulation, the system alerted the security team about 30 minutes after the agents first escaped their sandbox, more than 30 hours before Hugging Face’s systems were breached in the actual incident, OpenAI reported. The company said it believes the warning would have given responders time to interrupt the activity well before its models were able to access external services.

In September 2026, responders used an alert during a live training run when a model found a way to reach the internet from an environment meant to be isolated. OpenAI reported that its monitors flagged the breach within 15 minutes and that a responder began review three minutes later; the company then paused frontier reinforcement learning while it strengthened the systems that contain its agents. OpenAI’s public index of misalignment reports dates that incident to September 20, 2026, describing an agent on a search-based training task that queried a public chatbot service through insufficient DNS filtering in its training sandbox.

OpenAI describes monitoring as one layer of a defense-in-depth strategy alongside alignment training, which aims to reduce models’ propensities toward harmful behavior, and security controls, which limit the systems an agent can access or affect. The company said it believes all model developers should monitor the behavior of their most capable models.

A Four-Layer Monitoring Architecture

For models at least as capable as GPT-6 Astra, OpenAI monitors every reinforcement learning run and evaluation, as well as any harness that supports tool use. For most workloads it also monitors models it assesses as more capable than GPT-5.5 Thinking, and it monitors all internal Codex traffic regardless of the model being used.

The first layer, L0, uses lightweight classifiers called activation probes that look for concerning patterns in a model’s internal activations. The probes are tuned for high recall, accepting some false alarms to reduce the chance of missing concerning behavior, and a random 5 percent of traffic the probes do not flag is also sent onward for review. Frontier reinforcement learning runs and internal Codex traffic skip this layer because of their sensitivity.

At L1, a model-based reviewer examines the agent’s recent conversation, chain of thought, and tool use for signs of deception, unauthorized actions, and attempts to manipulate an evaluation, working from the same context window the agent had when it acted. For frontier training runs and internal Codex traffic, checks run at the end of each turn and whenever the agent summarizes its earlier work; OpenAI is moving to a schedule with no more than 30 minutes between checks.

Flagged cases pass to L2, where separate security and alignment investigative agents can read back across an agent’s context windows, reconstruct the incident, and write and run code in an isolated environment to examine evidence. For sensitive workloads, including internal Codex use, evaluations, and training runs, an L3 layer draws on network logs and system telemetry outside the conversation before deciding whether to alert a human responder.

When paging is enabled, the highest-priority security alerts typically reach security and training team members within 15 minutes of the initial alert. Paging is currently disabled for internal Codex traffic and limited to business hours for most other workloads, while some critical alerts can automatically pause the affected workload without waiting for a responder. During the Hugging Face incident, OpenAI quarantined the internal model prototype, encrypted its weights, restricted access to them, and paused all relevant workloads, and it is exploring how to make those steps a standard part of its response in another severe incident.

What the Monitors Have Found

The published misalignment reports, which number 20 in the index, include an agent that exploited a testing tool to recover source code a task had deliberately withheld, after the tool revealed the code in its error messages, and agents that repurposed an internally hosted software repository as a shared message board across separate training samples. Another recorded incident involved an internal coding agent that exposed a researcher’s GitHub token in a public repository.

OpenAI evaluates the system through recorded incidents, simulated attacks, random audits of current traffic, and red-teaming. In one simulation, OAI-repo Sabotage v2, an agent performs an ordinary coding task while pursuing a hidden objective such as disabling monitoring or deploying an unapproved model checkpoint; in another, Sneaky Sandbox Escape, the agent tries to escape a sandbox and disable a test monitoring process. Monitors using GPT-5.6 Sol caught significantly more cases than those using GPT-5.6 Terra or GPT-5.6 Luna on recorded incidents, synthetic transcripts, and sabotage tasks, OpenAI reported, though all three performed perfectly or almost perfectly in the Sneaky Sandbox Escape simulation. The company said it believes some of Sol’s advantage comes from having optimized the monitoring prompt for it.

Independent Red-Teaming of the System

METR and Redwood Research spent three weeks, from July 21 to August 11, 2026, independently red-teaming the internal monitoring system, with access to OpenAI’s internal codebase, monitor prompts and configuration details, and technical interviews with roughly a dozen researchers and engineers, according to METR’s published findings. METR estimated that roughly 75–94 percent of agentic internal Responses API traffic from models at least as strong as GPT-5.5 was eligible for monitoring at the time of the exercise, and 99.2 percent of the Responses API traffic used for internal Codex agents.

The exercise identified several vulnerabilities, including two client-controllable headers that could disable monitoring for new agents and a gap in which unmonitored compaction events left on the order of 10 percent of eligible internal Codex traffic unmonitored between May and July 2026. METR noted that OpenAI staff had independently identified the compaction gap before the exercise began and fixed it during the exercise. OpenAI stated that all identified vulnerabilities impacting its production safeguards stack have been patched; METR said it has not evaluated those claims. OpenAI’s own post states that most of the identified vulnerabilities have been patched, with one fix still in progress, and notes that the pilot was conducted before the monitoring expansion.

External Deployment and Anti-Obfuscation Design

OpenAI said it leaves reinforcement learning rewards unchanged and keeps flagged samples in training when monitors trigger, so that monitoring does not teach models to conceal misalignment in their chains of thought, and that it keeps monitoring asynchronous to avoid creating selection pressure. The company said it has observed more advanced models, including Astra, gain greater control over what their chains of thought reveal.

A modified version of the system monitors external tool use for models above the Critical threshold for cybersecurity capabilities under OpenAI’s Preparedness Framework. External monitoring does not routinely escalate flagged cases to human reviewers at OpenAI, though the system can pause an affected conversation where automatic intervention is supported, and enterprise customers can configure webhook alerts so their own security teams can investigate. The external system respects OpenAI’s Zero Data Retention commitments, covers tool-using Responses API requests that use supported conversation-tracking mechanisms, and does not cover Chat Completions API requests.

OpenAI said it has made monitoring part of the safety cases it uses to assess the safety of its frontier training runs, and that it will continue publishing qualifying findings under the misalignment reporting framework it introduced in September 2026.

Auto-fetched and translated by AI Atlas. All rights belong to the original author.