Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories

• URL: http://arxiv.org/abs/2606.03979v1
• Date: Tue, 02 Jun 2026 17:56:55 GMT
• Title: Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories
• Abstract summary: We introduce a ''Sleep'' paradigm that allows the models to continually learn and distill their short-term fragile memories into stable long-term knowledge with replay.<n>Inspired by human learning process, we present a new Generalized Distillation process for Knowledge Seeding (i.e., the combination of on-policy distillation with Reinforcement Learning)-based imitation learning.<n>Our experiments on long-horizon, continual learning, knowledge incorporation, and few-shot generalization tasks support the importance of the sleep stage.
• Score: 43.8851217839697
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The past few decades have witnessed significant advances in the design of machine learning algorithms, from early studies on task-specific shallow models to more general deep Large Language Models (LLMs). Despite showing promising results in tasks that require instant prediction or in-context learning, existing models lack the ability to continually learn and effectively transfer their temporal in-context knowledge to their long-term parameters. Inspired by human learning process, we introduce a ''Sleep'' paradigm that allows the models to continually learn, distill their short-term fragile memories into stable long-term knowledge with replay, and recursively improve themselves with ''Dreaming'' process. In more detail, sleep consists of two stages: (1) Memory Consolidation: an upward distillation process, called Knowledge Seeding, where the memories of a smaller-self are distilled into a larger network to provide more capacity while preserving the knowledge. As a proof of concept, we present a new Generalized Distillation process for {Knowledge Seeding} (i.e., the combination of on-policy distillation with Reinforcement Learning (RL)-based imitation learning); (2) Dreaming: a self-improvement phase, where the model uses RL to generate a curriculum of synthetic data to rehearse new knowledge and refine existing capabilities without human supervision. Our experiments on long-horizon, continual learning, knowledge incorporation, and few-shot generalization tasks support the importance of the sleep stage.

Related papers

• In Two Minds about Lifelong Learning: Exploring Hemispheric Redundancy and Specialisation in Neural Models [6.404762820001178]
  This paper attempts to build a high-level framework for how animals learn and preserve knowledge.<n>We focus on three concepts: experience replay, REM sleep, and bilaterality.<n>We propose 4MAS, a novel macroarchitecture demonstrating how machine learning models might benefit from asymmetric hemispheres.
  arXiv  Detail & Related papers  (2026-08-20T00:15:18Z)
• Nested Learning: The Illusion of Deep Learning Architectures [57.41377373511876]
  We present a new learning paradigm, called Nested Learning (NL), that coherently represents a machine learning model with a set of nested, multi-level, and/or parallel problems.<n>We show three core contributions: Expressive generalizations are in fact as generalizations with deep memory and/or more powerful learning rules.<n>We present a new continuum for memory system that generalizes the traditional viewpoint of long/short-term memory.
  arXiv  Detail & Related papers  (2025-12-31T07:59:43Z)
• Semi-parametric Memory Consolidation: Towards Brain-like Deep Continual Learning [59.35015431695172]
  We propose a novel biomimetic continual learning framework that integrates semi-parametric memory and the wake-sleep consolidation mechanism.<n>For the first time, our method enables deep neural networks to retain high performance on novel tasks while maintaining prior knowledge in real-world challenging continual learning scenarios.
  arXiv  Detail & Related papers  (2025-04-20T19:53:13Z)
• Retentive or Forgetful? Diving into the Knowledge Memorizing Mechanism of Language Models [49.39276272693035]
  Large-scale pre-trained language models have shown remarkable memorizing ability. Vanilla neural networks without pre-training have been long observed suffering from the catastrophic forgetting problem. We find that 1) Vanilla language models are forgetful; 2) Pre-training leads to retentive language models; 3) Knowledge relevance and diversification significantly influence the memory formation.
  arXiv  Detail & Related papers  (2023-05-16T03:50:38Z)
• Lifelong Person Re-Identification via Knowledge Refreshing and Consolidation [35.43406281230279]
  Key challenge for Lifelong person re-identification (LReID) is how to incrementally preserve old knowledge and gradually add new capabilities to the system. Inspired by the biological process of human cognition where the somatosensory neocortex and the hippocampus work together in memory consolidation, we formulated a model called Knowledge Refreshing and Consolidation (KRC) KRC achieves both positive forward and backward transfer. More specifically, a knowledge refreshing scheme is incorporated with the knowledge rehearsal mechanism to enable bi-directional knowledge transfer.
  arXiv  Detail & Related papers  (2022-11-29T13:39:45Z)
• Anti-Retroactive Interference for Lifelong Learning [65.50683752919089]
  We design a paradigm for lifelong learning based on meta-learning and associative mechanism of the brain. It tackles the problem from two aspects: extracting knowledge and memorizing knowledge. It is theoretically analyzed that the proposed learning paradigm can make the models of different tasks converge to the same optimum.
  arXiv  Detail & Related papers  (2022-08-27T09:27:36Z)
• Bilevel Continual Learning [76.50127663309604]
  We present a novel framework of continual learning named "Bilevel Continual Learning" (BCL) Our experiments on continual learning benchmarks demonstrate the efficacy of the proposed BCL compared to many state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-07-30T16:00:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.