Finding Structure in Continual Learning

• URL: http://arxiv.org/abs/2602.04555v1
• Date: Wed, 04 Feb 2026 13:44:26 GMT
• Title: Finding Structure in Continual Learning
• Abstract summary: We propose a reformulation of the continual learning objective using Douglas-Rachford Splitting (DRS)<n>This reframes the learning process not as a direct trade-off, but as a negotiation between two decoupled objectives.<n>Our approach achieves an efficient balance between stability and plasticity without the need for auxiliary modules or complex add-ons.
• Score: 25.94714152174335
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Learning from a stream of tasks usually pits plasticity against stability: acquiring new knowledge often causes catastrophic forgetting of past information. Most methods address this by summing competing loss terms, creating gradient conflicts that are managed with complex and often inefficient strategies such as external memory replay or parameter regularization. We propose a reformulation of the continual learning objective using Douglas-Rachford Splitting (DRS). This reframes the learning process not as a direct trade-off, but as a negotiation between two decoupled objectives: one promoting plasticity for new tasks and the other enforcing stability of old knowledge. By iteratively finding a consensus through their proximal operators, DRS provides a more principled and stable learning dynamic. Our approach achieves an efficient balance between stability and plasticity without the need for auxiliary modules or complex add-ons, providing a simpler yet more powerful paradigm for continual learning systems.

Related papers

• When Does Continual Learning Require Learning [68.55508958298942]
  We show that continual learning is not a single capability.<n>Different patterns of environmental change require fundamentally different update behaviors.<n>We hope that understanding where each method succeeds and fails will guide the design of stronger continual learning systems.
  arXiv  Detail & Related papers  (2026-07-08T18:27:40Z)
• Task Switching Without Forgetting via Proximal Decoupling [43.88767637201092]
  In continual learning, the primary challenge is to learn new information without forgetting old knowledge.<n>A common solution addresses this trade-off through regularization, penalizing changes to parameters critical for previous tasks.<n>We propose a different approach that separates task learning from stability enforcement via operator splitting.
  arXiv  Detail & Related papers  (2026-04-20T21:28:13Z)
• Retrofit: Continual Learning with Bounded Forgetting for Security Applications [25.185616916987158]
  We propose RETROFIT, a data retrospective-free continual learning method that achieves bounded forgetting for effective knowledge transfer.<n>To mitigate interference, we apply low-rank and sparse updates that confine parameter changes to independent subspaces.<n>In malware detection under temporal drift, it substantially improves the retention score, from 20.2% to 38.6% over CL baselines, and exceeds the oracle upper bound on new data.
  arXiv  Detail & Related papers  (2025-11-14T16:07:03Z)
• Continual Learning in Vision-Language Models via Aligned Model Merging [84.47520899851557]
  We present a new perspective based on model merging to maintain stability while still retaining plasticity.<n>To maximize the effectiveness of the merging process, we propose a simple mechanism that promotes learning aligned weights with previous ones.
  arXiv  Detail & Related papers  (2025-05-30T20:52:21Z)
• Reinforced Interactive Continual Learning via Real-time Noisy Human Feedback [59.768119380109084]
  This paper introduces an interactive continual learning paradigm where AI models dynamically learn new skills from real-time human feedback.<n>We propose RiCL, a Reinforced interactive Continual Learning framework leveraging Large Language Models (LLMs)<n>Our RiCL approach substantially outperforms existing combinations of state-of-the-art online continual learning and noisy-label learning methods.
  arXiv  Detail & Related papers  (2025-05-15T03:22:03Z)
• Pareto Continual Learning: Preference-Conditioned Learning and Adaption for Dynamic Stability-Plasticity Trade-off [25.33978152436581]
  We propose a novel framework that reformulates the stability-plasticity trade-off in continual learning.<n>ParetoCL can be seen as an objective augmentation approach that learns from different objective combinations of stability and plasticity.
  arXiv  Detail & Related papers  (2025-03-30T10:38:36Z)
• Lifelong Learning with Task-Specific Adaptation: Addressing the Stability-Plasticity Dilemma [13.567823451714405]
  Lifelong learning aims to continuously acquire new knowledge while retaining previously learned knowledge.<n>The stability-plasticity dilemma requires models to balance the preservation of previous knowledge (stability) with the ability to learn new tasks (plasticity)<n>This paper proposes AdaLL, an adapter-based framework designed to address the dilemma through a simple, universal, and effective strategy.
  arXiv  Detail & Related papers  (2025-03-08T13:33:38Z)
• Temporal-Difference Variational Continual Learning [77.92320830700797]
  We propose new learning objectives that integrate the regularization effects of multiple previous posterior estimations.<n>Our approach effectively mitigates Catastrophic Forgetting, outperforming strong Variational CL methods.
  arXiv  Detail & Related papers  (2024-10-10T10:58:41Z)
• ReconBoost: Boosting Can Achieve Modality Reconcilement [89.4377895465204]
  We study the modality-alternating learning paradigm to achieve reconcilement. We propose a new method called ReconBoost to update a fixed modality each time. We show that the proposed method resembles Friedman's Gradient-Boosting (GB) algorithm, where the updated learner can correct errors made by others.
  arXiv  Detail & Related papers  (2024-05-15T13:22:39Z)
• Recall-Oriented Continual Learning with Generative Adversarial Meta-Model [5.710971447109951]
  We propose a recall-oriented continual learning framework to address the stability-plasticity dilemma. Inspired by the human brain's ability to separate the mechanisms responsible for stability and plasticity, our framework consists of a two-level architecture. We show that our framework not only effectively learns new knowledge without any disruption but also achieves high stability of previous knowledge.
  arXiv  Detail & Related papers  (2024-03-05T16:08:59Z)
• Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization [51.34904967046097]
  Continual learning seeks to overcome the challenge of catastrophic forgetting, where a model forgets previously learnt information. We introduce a novel prior-based method that better constrains parameter growth, reducing catastrophic forgetting. Results show that BAdam achieves state-of-the-art performance for prior-based methods on challenging single-headed class-incremental experiments.
  arXiv  Detail & Related papers  (2023-09-15T17:10:51Z)
• Incorporating Neuro-Inspired Adaptability for Continual Learning in Artificial Intelligence [59.11038175596807]
  Continual learning aims to empower artificial intelligence with strong adaptability to the real world. Existing advances mainly focus on preserving memory stability to overcome catastrophic forgetting. We propose a generic approach that appropriately attenuates old memories in parameter distributions to improve learning plasticity.
  arXiv  Detail & Related papers  (2023-08-29T02:43:58Z)
• Lifelong Person Re-Identification via Knowledge Refreshing and Consolidation [35.43406281230279]
  Key challenge for Lifelong person re-identification (LReID) is how to incrementally preserve old knowledge and gradually add new capabilities to the system. Inspired by the biological process of human cognition where the somatosensory neocortex and the hippocampus work together in memory consolidation, we formulated a model called Knowledge Refreshing and Consolidation (KRC) KRC achieves both positive forward and backward transfer. More specifically, a knowledge refreshing scheme is incorporated with the knowledge rehearsal mechanism to enable bi-directional knowledge transfer.
  arXiv  Detail & Related papers  (2022-11-29T13:39:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.