2026年10月27日 (英语).
Ali Behrouz和Vahab Mirrokni, Google研究
我们在持续学习中引入了睡眠阶段，使大型语言模型能够不断融合新事实，增强推理，同时减轻灾难性的遗忘或静态的知识断层。
现代LLMs通过预训和内文学习(ICL)取得了令人印象深刻的表现。 然而，一旦部署，它们基本上保持静止状态，在固定的知识截止日期运作，使其无法获得长期能力或适应新信息。
虽然用新数据更新模型似乎是显而易见的解决办法，但持续的参数更新往往引发灾难性的遗忘，而学习新任务将严重削弱以往学习过的绩效。 或者，严格依靠ICL将知识保留限制在直接上下文窗口：一旦上下文会话结束，收购的短期信息就会被遗忘。
受人类大脑如何在休息期间处理和稳定记忆的启发，我们引入了持续学习中的睡眠框架：一种学习范式，使大型语言模型(LLMs)具有连续的周期性生命周期，使他们能够不断学习，将脆弱的短期环境整合为稳定的长期参数。 通过这个框架，我们表明，在主动交互和离线内存合并之间的交替，LLMs可以不断获取新的事实，保留长达10M令牌的长文知识，并改善复杂的推理。
传统的机器学习依赖于严格区分培训和推论。 在不断学习的框架内，这种区分变得人为的；一个聪明的agent必须不断学习，适应和处理信息。
在睡眠模式下，LLM运行于两个初级州：
- 活动(醒悟)阶段：模型积极接收，处理，并与输入进行交互，将临时信息存储在快速，短期内存组件中。
- 睡眠阶段：模型暂停处理新的输入，并将其内部计算转向内存合并和自我改进。
虽然"睡眠","梦想","自我改善"等术语借用神经科学，但我们的框架完全在固定的模型架构和人定义的训练规则中运作。 在离线睡眠阶段，模型解锁预分配的Mixture-of-Experts(MOE)内存块，并使用自生成的实践实例更新其数值权重。 这个自动化的训练后周期有助于部署的模型吸收新的事实，提高其推理精度，同时保持核心代码基础和设计不变。
睡眠范式通过两个阶段解决灾难性的遗忘和持续的适应问题：记忆巩固和梦想。
在人类大脑中，非快速眼动(NREM)慢波睡眠通过抽象和蒸馏将脆弱的记忆整合为长期结构。 我们提议的睡眠框架通过两个关键技术机制反映了这种自然现象：
- 参数扩展和连续内存结构：为了防止灾难性的遗忘，模型在睡眠时动态地缩小其容量。 新解锁的低级专家采用专家混合结构(MoE)和希望架构(Hope architecture)整合为稳定，低频的内存块，在不干扰旧参数的情况下存储传入的知识。
- 知识种子(上衍射)：储存在高频层的脆弱短期记忆(如内隐记忆)被转移到新扩展的长期参数中。 这是通过通用知识消化(GKD)与基于强化学习(RL)的模仿学习相结合实现的：
- 政策蒸馏：扩大的学生模式从自采样序列产生的教师分发中学习特定符号反馈。
- 学习模仿(LTI)：学生模型在完成教师生成的文本的前缀时，根据语义学和绝对相似性(Levenshtein star)得到奖励。
– 突触穿梭：内存整合为低频模块完成后，高频临时参数被重置，为未来活动会话腾出能力。
在REM睡眠期间，人类大脑积极模拟各种情景，巩固情感/语义联系，探索新的神经途径。 反之，在“梦想阶段”中，睡眠框架执行一个自主的自我改进循环：
- 随机专家选择：在合成数据生成过程中，MOE路由器随机激活非标准专家。 这迫使模型将不同的知识领域混为一谈，并发现潜在的模式。
- 解析排练：该模型生成合成训练样品("梦想")，并使用梯度数据选择进行过滤，以优先排列具有最大潜力的样品，提升整体模型能力。
- RL精细化：该模型在顶级的梦样上执行监督的微调，并评价该一代是否改善了下游任务的表现。
除了实证结果外，我们的睡眠范式还基于一个理论，该理论澄清了为什么其两个阶段(记忆巩固和梦想)能够支持持续学习，而不会简单地用新知识换取旧能力。
对于内存整合，分析显示，知识种子可以理解为结构化，低级压缩问题。 该模型没有被要求将它短期痕迹的所有细节复制到它的参数中。 相反，它确定了一套紧凑的方向，以最好地保存模型本身的政策行为和学习目标所奖励的语义素质。
这一视角解释了两个信号的互补作用：政策提炼与模型在推论时实际产生的轨迹一致，减少只在教师生成的分布方面进行培训时可能出现的不匹配现象；同时，语义奖励改变了值得保留的方向，确定了保留意义和任务相关行为的方向，而不仅仅是匹配象征性概率。 然后，参数扩展提供了编写这些选定的记忆而无需覆盖先前合并的知识所需的能力。
对于梦境，该理论将随机专家激活和基于奖励的样本选择定性为改进学习几何机制。 注入不同的MOE拓宽了合成生成过程中探索的梯度方向，使模型的更新收集比普通的守夜数据本身所引发的更新更加丰富。 渐变式滤波然后强调带有最有用的学习信号的合成经验，加强信息化方向，同时保留基本梯度空间的结构。
这些步骤共同产生一个更加均衡的优化景观：学习较少被少数方向所支配，梯度条件较好，模型在完善能力时可以取得更稳定的进展。 在这种观点中，梦想不仅仅是自我生成的数据增强；它是一个有意使模型自我改进的更新多样化、选择和整合的过程。
就连续翻译而言，如满洲语和卡拉芒语，睡眠框架在多阶段评价中维持了近零的性能下降，而ICL、Cartridges和受监督的微调(SFT)等标准基线则在连续学习过程中遭受严重的灾难性遗忘。
在对BABILong 基准测试的长文理解中，我们的睡眠方法展示了近乎完美的放大到10M令牌，表现优于GPT-4等强基线，泰坦斯通过使用睡眠时间合并将长文信息直接压缩成参数内存。
关于在SQuAD和持续预训(CPT)上评估的事实知识纳入，我们的睡眠方法在单任务上达到了48.9%，在CPT评价上达到了46.2%，表现优于基准方法，包括基础模型、SFT和自适应语言模型(SEAL)，成功地整合了新的事实信息，同时不贬低了现有的模型表述。
最后，根据数学推理基准测试包括AIME 24/25和HMMT 25，睡眠增强基模分别实现了79.2%,69.0%和46.1%的精度分数。 这种表现超过了SFT，组类相对政策优化(GRPO)，以及政策上自我消化(OPSD)等基线，表明将内存整合与仿真学习进行配对会驱动优异的推理性能。
我们的睡眠模式提供了一条通往自主、不断学习人工智能系统的道路。 通过超越静态的预训和脆弱的上下文windows，在主动学习和离线休息之间交替，LLMs可以使LLMs保留短期记忆，抽象的新技能进入长期参数结构，并不断自我改进而不会灾难性的遗忘。
这项研究由阿里·贝赫鲁兹(Google和康奈尔大学)，法尔诺什·哈希姆(科内尔大学)，阿德尔·贾万马尔德(Google和南加州大学)和瓦哈布·米尔罗克尼进行。 我们感谢袁登和科林纳·科尔特斯审查这项工作及其宝贵建议。 我们也感谢马克·辛伯格(Mark Simborg)，金伯利·施威德(Kimberly Schwede)
×
❮
❯
×
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
October 27, 2026
Ali Behrouz and Vahab Mirrokni, Google Research
We introduce a sleep phase in continual learning, enabling large language models to continuously integrate new facts and enhance reasoning while mitigating catastrophic forgetting or static knowledge cutoffs.
Modern LLMs achieve impressive performance via pre-training and in-context learning (ICL). However, once deployed, they remain largely static, operating with a fixed knowledge cutoff date that leaves them unable to acquire long-term capabilities or adapt to new information.
While updating models with new data seems like the obvious solution, continual parameter updates often trigger catastrophic forgetting, a fundamental issue where learning new tasks severely degrades performance on previously learned ones. Alternatively, relying strictly on ICL limits knowledge retention to the immediate context window: once the context session ends, the acquired short-term information is forgotten.
Inspired by how the human brain processes and stabilizes memory during rest, we introduce a sleep framework in continual learning: a learning paradigm that equips large language models (LLMs) with a continuous periodic lifecycle, enabling them to continually learn, consolidate fragile short-term context into stable long-term parameters. With this framework, we demonstrate that alternating between active interaction and offline memory consolidation allows LLMs to continuously acquire new facts, retain long-context knowledge up to 10M tokens, and improve complex reasoning.
Traditional machine learning relies on a strict separation between training and inference. In a continual learning framework, this distinction becomes artificial; an intelligent agent must constantly learn, adapt, and process information.
Under the sleep paradigm, an LLM operates across two primary states:
- Active (wake) phase: The model actively receives, processes, and interacts with inputs, storing temporary information in fast, short-term memory components.
- Sleep phase: The model pauses processing new inputs and redirects its internal computations toward memory consolidation and self-improvement.
While terms like "sleep," "dreaming," and "self-improvement" borrow from neuroscience, our framework operates entirely within a fixed model architecture and human-defined training rules. During the offline sleep phase, the model unlocks pre-allocated Mixture-of-Experts (MoE) memory blocks and updates its numerical weights using self-generated practice examples. This automated post-training cycle helps a deployed model absorb new facts and sharpen its reasoning accuracy while keeping its core codebase and design unchanged.
The sleep paradigm addresses catastrophic forgetting and continuous adaptation through two stages: memory consolidation and dreaming.
In the human brain, non-rapid eye movement (NREM) slow-wave sleep consolidates fragile memories into long-term structures via abstraction and distillation. Our proposed sleep framework mirrors this natural phenomenon through two key technical mechanisms:
- Parameter expansion & continuum memory hierarchy: To prevent catastrophic forgetting, the model dynamically scales its capacity during sleep. Using a Mixture of Experts (MoE) and Hope architecture , newly unlocked, low-rank experts are integrated into stable, lower-frequency memory blocks to store incoming knowledge without disrupting older parameters.
- Knowledge seeding (upward distillation): Fragile short-term memories stored in high-frequency layers (e.g., in-context memory) are transferred into the newly expanded long-term parameters. This is achieved through Generalized Knowledge Distillation (GKD) combined with reinforcement learning (RL)-based imitation learning:
- On-policy distillation: The expanded student model learns token-specific feedback from teacher distributions generated on self-sampled sequences.
- Learning to imitate (LTI): The student model is rewarded based on semantic and absolute similarity (Levenshtein distance) when completing prefixes of teacher-generated text.
- Synaptic pruning: Once the memory consolidation into lower-frequency modules is complete, the high-frequency temporary parameters are reset, freeing up capacity for future active sessions.
During REM sleep, the human brain actively simulates scenarios, consolidates emotional/semantic connections, and explores novel neural pathways. Mirroring this, during the “dreaming stage”, the sleep framework implements an autonomous self-improvement loop:
- Random expert selection: During synthetic data generation, the MoE router randomly activates non-standard experts. This forces the model to mix disparate domains of knowledge and uncover latent patterns.
- Curated rehearsal: The model generates synthetic training samples ("dreams") and filters them using gradient-based data selection to prioritize samples with the highest potential to boost overall model capability.
- RL refinement: The model performs supervised fine-tuning on top-rated dream samples and evaluates whether the generation improves performance on downstream tasks.
Beyond the empirical results, our sleep paradigm is grounded in a theory that clarifies why its two stages (memory consolidation and dreaming) can support continual learning without simply trading new knowledge for old capabilities.
For memory consolidation, the analysis shows that knowledge seeding can be understood as a structured, low-rank compression problem. The model is not asked to copy every detail of its short-term traces into its parameters. Instead, it identifies a compact set of directions that best preserve both the model’s own on-policy behavior and the semantic qualities rewarded by the learning objective.
This perspective explains the complementary roles of the two signals: on-policy distillation aligns consolidation with the trajectories the model will actually produce at inference time, reducing the mismatch that can arise when training only on teacher-generated distributions; meanwhile, the semantic reward changes what is worth retaining, prioritizing directions that preserve meaning and task-relevant behavior rather than merely matching token probabilities. Parameter expansion then provides the capacity needed to write these selected memories without overwriting previously consolidated knowledge.
For dreaming, the theory characterizes random expert activation and reward-based sample selection as mechanisms for improving the geometry of learning. Injecting different MoEs broadens the set of gradient directions explored during synthetic generation, exposing the model to a richer collection of updates than those induced by its ordinary wake-time data alone. Gradient-based filtering then emphasizes the synthetic experiences that carry the most useful learning signal, strengthening informative directions while preserving the structure of the underlying gradient space.
Together, these steps yield a more balanced optimization landscape: learning is less dominated by a small number of directions, gradients are better conditioned, and the model can make more stable progress when refining its capabilities. In this view, dreaming is not simply self-generated data augmentation; it is a deliberate process for diversifying, selecting, and consolidating the updates through which the model improves itself.
For continual translation, in unseen languages, such as Manchu and Kalamang, the sleep framework maintained a near-zero performance drop in multi-stage evaluations, whereas standard baselines like ICL, Cartridges, and supervised fine-tuning (SFT) suffered severe catastrophic forgetting during sequential learning.
In long-context understanding on the BABILong benchmark, our sleep method demonstrated near-perfect scaling up to 10M tokens, outperforming strong baselines like GPT-4, and Titans by using sleep-time consolidation to compress long-context information directly into parametric memory.
For factual knowledge incorporation evaluated on SQuAD and continued pre-training (CPT), our sleep method achieved 48.9% on single-task and 46.2% on CPT evaluations, outperforming baseline methods including the base model, SFT, and Self-Adapting Language Models (SEAL) by successfully integrating new factual information without degrading existing model representations.
Finally, on mathematical reasoning benchmarks including AIME 24/25 and HMMT 25, sleep-enhanced base-model achieved accuracy scores of 79.2%, 69.0%, and 46.1%, respectively. This outperformed baselines like SFT, group relative policy optimization (GRPO), and on-policy self-distillation (OPSD), showing that pairing memory consolidation with imitation learning drives superior reasoning performance.
Our sleep paradigm provides a path toward autonomous, continually learning AI systems. By moving beyond static pre-training and fragile context windows, alternating between active learning and offline rest empowers LLMs to retain short-term memories, abstract new skills into long-term parameter structures, and continuously self-improve without catastrophic forgetting.
This research was conducted by Ali Behrouz (Google and Cornell University), Farnoosh Hashemi (Cornell University), Adel Javanmard (Google and University of Southern California), and Vahab Mirrokni. We thank Yuan Deng and Corinna Cortes for reviewing the work and their valuable suggestions. We also thank Mark Simborg, Kimberly Schwede, and Tim Herrmann for their help in crafting this blog post.
×
❮
❯
×
Auto-fetched and translated by AI Atlas. All rights belong to the original author.