Theoretical Foundations of Continual Learning via Drift-Plus-Penalty

• URL: http://arxiv.org/abs/2606.08452v1
• Date: Sun, 07 Jun 2026 04:51:32 GMT
• Title: Theoretical Foundations of Continual Learning via Drift-Plus-Penalty
• Abstract summary: Continual learning (CL) addresses this challenge by incorporating new tasks while mitigating catastrophic forgetting.<n>We introduce a control-theoretic perspective on CL that explicitly regulates the evolution of forgetting.<n>We propose COntinual Learning with Drift-PlusPenalty (COLD), a continual framework based on the Drift-PlusPenalty principle from optimization.
• Score: 6.614755043607776
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In many real-world settings, data streams are nonstationary and arrive sequentially, requiring learning systems to adapt continuously without retraining from scratch. Continual learning (CL) addresses this challenge by incorporating new tasks while mitigating catastrophic forgetting, where learning new information degrades performance on previously acquired knowledge. We introduce a control-theoretic perspective on CL that explicitly regulates the evolution of forgetting, framing adaptation as a controlled process subject to long-term stability constraints. We focus on replay-based CL, where a finite memory buffer stores representative samples from prior tasks. We propose COntinual Learning with Drift-Plus-Penalty (COLD), a continual learning framework based on the Drift-Plus-Penalty (DPP) principle from stochastic optimization. To facilitate analysis, we also consider an oracle variant, COLD-ORACLE, as a reference benchmark. At each task, both methods minimize the current task loss while maintaining a virtual queue that tracks deviations from long-term stability on previously learned tasks, capturing the stability-plasticity trade-off as a regulated dynamical process. We establish stability and convergence guarantees that characterize this trade-off through a tunable control parameter. Experiments on standard benchmarks demonstrate that COLD consistently outperforms a broad range of state-of-the-art CL methods while providing competitive and controllable forgetting behavior through explicit regulation of stability and plasticity.

Related papers

• Federated Continual Learning as a Distributed Drift-Plus-Penalty Control Problem [12.498691027665332]
  Federated Continual Learning (FCL) is fundamental to real-world distributed learning systems.<n>FCL requires models to adapt to sequential, non-IID data across clients while mitigating catastrophic forgetting and client drift.<n>We propose Federated Queue-regulated Continual Learning (FedQCL), a framework based on Lynov drift-plus-penalty (DPP) optimization.
  arXiv  Detail & Related papers  (2026-08-21T18:21:19Z)
• To Retain or to Adapt? Generalizing Continual Learning [39.95178050304739]
  We argue that in non-stationary environments prioritizing retention can impede real-time adaptation.<n>We formalize CL as an online optimization problem governed by the interaction between environmental and learning dynamics.<n>We propose a new general class of continual learning algorithms, which we call Predictive Continual Learning.
  arXiv  Detail & Related papers  (2026-07-06T20:11:53Z)
• Revitalizing the Beginning: Avoiding Storage Dependency for Model Merging in Continual Learning [57.10440766103372]
  Trajectory Regularized Merging (TRM) is a framework that reformulates the merging phase as an optimization process within an augmented trajectory subspace.<n>Our framework integrates three synergistic objectives including task alignment, prediction consistency, and gradient responsiveness to concurrently preserve merged model's historical stability and re-activate optimization dynamics.
  arXiv  Detail & Related papers  (2026-05-08T14:07:32Z)
• Stabilizing Reinforcement Learning with LLMs: Formulation and Practices [61.361819972410046]
  We show why and under what conditions the true sequence-level reward can be optimized via a surrogate token-level objective in policy gradient methods such as REINFORCE.<n>This insight provides a principled explanation for the crucial role of several widely adopted techniques in stabilizing RL training.
  arXiv  Detail & Related papers  (2025-12-01T07:45:39Z)
• Iterative Refinement of Flow Policies in Probability Space for Online Reinforcement Learning [56.47948583452555]
  We introduce the Stepwise Flow Policy (SWFP) framework, founded on the key insight that discretizing the flow matching inference process via a fixed-step Euler scheme aligns it with the variational Jordan-Kinderlehrer-Otto principle from optimal transport.<n>SWFP decomposes the global flow into a sequence of small, incremental transformations between proximate distributions.<n>This decomposition yields an efficient algorithm that fine-tunes pre-trained flows via a cascade of small flow blocks, offering significant advantages.
  arXiv  Detail & Related papers  (2025-10-17T07:43:51Z)
• Adapt before Continual Learning [9.477667054965782]
  Adapting PTMs before the core CL process (ACL) is a novel framework that introduces a plug-and-play adaptation phase prior to learning each new task.<n>ACL significantly improves CL performance across benchmarks and integrated methods.
  arXiv  Detail & Related papers  (2025-06-04T13:46:33Z)
• Online Curvature-Aware Replay: Leveraging $\mathbf{2^{nd}}$ Order Information for Online Continual Learning [2.0165668334347187]
  We formalize replay-based online joint optimization with explicit KL-divergence constraints on replay data.<n>We show how to adapt the estimation of the FIM to a continual setting second-order optimization for non-iid data.<n>OCAR outperforms state-of-the-art methods in continual metrics achieving higher average accuracy throughout the training process in three different benchmarks.
  arXiv  Detail & Related papers  (2025-02-03T22:31:36Z)
• Continual Task Learning through Adaptive Policy Self-Composition [54.95680427960524]
  CompoFormer is a structure-based continual transformer model that adaptively composes previous policies via a meta-policy network. Our experiments reveal that CompoFormer outperforms conventional continual learning (CL) methods, particularly in longer task sequences.
  arXiv  Detail & Related papers  (2024-11-18T08:20:21Z)
• Temporal-Difference Variational Continual Learning [77.92320830700797]
  We propose new learning objectives that integrate the regularization effects of multiple previous posterior estimations.<n>Our approach effectively mitigates Catastrophic Forgetting, outperforming strong Variational CL methods.
  arXiv  Detail & Related papers  (2024-10-10T10:58:41Z)
• Large Continual Instruction Assistant [59.585544987096974]
  Continual Instruction Tuning (CIT) is adopted to instruct Large Models to follow human intent data by data.<n>Existing update gradient would heavily destroy the performance on previous datasets during CIT process.<n>We propose a general continual instruction tuning framework to address the challenge.
  arXiv  Detail & Related papers  (2024-10-08T11:24:59Z)
• Keep Moving: identifying task-relevant subspaces to maximise plasticity for newly learned tasks [0.22499166814992438]
  Continual learning algorithms strive to acquire new knowledge while preserving prior information. Often, these algorithms emphasise stability and restrict network updates upon learning new tasks. But is all change detrimental? We propose that activation spaces in neural networks can be decomposed into two subspaces.
  arXiv  Detail & Related papers  (2023-10-07T08:54:43Z)
• Recurrent Neural Network Controllers Synthesis with Stability Guarantees for Partially Observed Systems [6.234005265019845]
  We consider the important class of recurrent neural networks (RNN) as dynamic controllers for nonlinear uncertain partially-observed systems. We propose a projected policy gradient method that iteratively enforces the stability conditions in the reparametrized space. Numerical experiments show that our method learns stabilizing controllers while using fewer samples and achieving higher final performance compared with policy gradient.
  arXiv  Detail & Related papers  (2021-09-08T18:21:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.