Pareto Continual Learning: Preference-Conditioned Learning and Adaption for Dynamic Stability-Plasticity Trade-off
- URL: http://arxiv.org/abs/2503.23390v1
- Date: Sun, 30 Mar 2025 10:38:36 GMT
- Title: Pareto Continual Learning: Preference-Conditioned Learning and Adaption for Dynamic Stability-Plasticity Trade-off
- Authors: Song Lai, Zhe Zhao, Fei Zhu, Xi Lin, Qingfu Zhang, Gaofeng Meng,
- Abstract summary: We propose a novel framework that reformulates the stability-plasticity trade-off in continual learning.<n>ParetoCL can be seen as an objective augmentation approach that learns from different objective combinations of stability and plasticity.
- Score: 25.33978152436581
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Continual learning aims to learn multiple tasks sequentially. A key challenge in continual learning is balancing between two objectives: retaining knowledge from old tasks (stability) and adapting to new tasks (plasticity). Experience replay methods, which store and replay past data alongside new data, have become a widely adopted approach to mitigate catastrophic forgetting. However, these methods neglect the dynamic nature of the stability-plasticity trade-off and aim to find a fixed and unchanging balance, resulting in suboptimal adaptation during training and inference. In this paper, we propose Pareto Continual Learning (ParetoCL), a novel framework that reformulates the stability-plasticity trade-off in continual learning as a multi-objective optimization (MOO) problem. ParetoCL introduces a preference-conditioned model to efficiently learn a set of Pareto optimal solutions representing different trade-offs and enables dynamic adaptation during inference. From a generalization perspective, ParetoCL can be seen as an objective augmentation approach that learns from different objective combinations of stability and plasticity. Extensive experiments across multiple datasets and settings demonstrate that ParetoCL outperforms state-of-the-art methods and adapts to diverse continual learning scenarios.
Related papers
- BECAME: BayEsian Continual Learning with Adaptive Model MErging [21.642774366793997]
We introduce a two-stage framework named BECAME, which synergizes the expertise of gradient projection and adaptive merging.
Our approach outperforms state-of-the-art CL methods and existing merging strategies.
arXiv Detail & Related papers (2025-04-03T15:07:28Z) - Continual Multimodal Contrastive Learning [70.60542106731813]
Multimodal contrastive learning (MCL) advances in aligning different modalities and generating multimodal representations in a joint space.
However, a critical yet often overlooked challenge remains: multimodal data is rarely collected in a single process, and training from scratch is computationally expensive.
In this paper, we formulate CMCL through two specialized principles of stability and plasticity.
We theoretically derive a novel optimization-based method, which projects updated gradients from dual sides onto subspaces where any gradient is prevented from interfering with the previously learned knowledge.
arXiv Detail & Related papers (2025-03-19T07:57:08Z) - Lifelong Learning with Task-Specific Adaptation: Addressing the Stability-Plasticity Dilemma [13.567823451714405]
Lifelong learning aims to continuously acquire new knowledge while retaining previously learned knowledge.<n>The stability-plasticity dilemma requires models to balance the preservation of previous knowledge (stability) with the ability to learn new tasks (plasticity)<n>This paper proposes AdaLL, an adapter-based framework designed to address the dilemma through a simple, universal, and effective strategy.
arXiv Detail & Related papers (2025-03-08T13:33:38Z) - Differentiable Information Enhanced Model-Based Reinforcement Learning [48.820039382764]
Differentiable environments have heralded new possibilities for learning control policies by offering rich differentiable information.<n>Model-based reinforcement learning (MBRL) methods exhibit the potential to effectively harness the power of differentiable information for recovering the underlying physical dynamics.<n>However, this presents two primary challenges: effectively utilizing differentiable information to 1) construct models with more accurate dynamic prediction and 2) enhance the stability of policy training.
arXiv Detail & Related papers (2025-03-03T04:51:40Z) - Neural Networks Remember More: The Power of Parameter Isolation and Combination [3.2430260063115233]
Catastrophic forgetting is a pervasive issue for pre-trained language models.<n>Key to solving this problem is to find a trade-off between the plasticity and stability of the model.<n>We propose a novel method to achieve a balance between model stability and plasticity.
arXiv Detail & Related papers (2025-02-16T02:58:57Z) - Temporal-Difference Variational Continual Learning [89.32940051152782]
A crucial capability of Machine Learning models in real-world applications is the ability to continuously learn new tasks.
In Continual Learning settings, models often struggle to balance learning new tasks with retaining previous knowledge.
We propose new learning objectives that integrate the regularization effects of multiple previous posterior estimations.
arXiv Detail & Related papers (2024-10-10T10:58:41Z) - EVCL: Elastic Variational Continual Learning with Weight Consolidation [14.485182089870928]
Continual learning aims to allow models to learn new tasks without forgetting what has been learned before.
This work introduces Elastic Variational Continual Learning with Weight Consolidation (E), a novel hybrid model that integrates the variational posterior approximation mechanism of Variational Continual Learning (EWC) with the regularization-based parameter-protection strategy of Elastic Weight Consolidation (EWC)
E effectively mitigates catastrophic forgetting and enables better capture of dependencies between model parameters and task-specific data.
arXiv Detail & Related papers (2024-06-23T00:32:06Z) - Evaluating and Improving Continual Learning in Spoken Language
Understanding [58.723320551761525]
We propose an evaluation methodology that provides a unified evaluation on stability, plasticity, and generalizability in continual learning.
By employing the proposed metric, we demonstrate how introducing various knowledge distillations can improve different aspects of these three properties of the SLU model.
arXiv Detail & Related papers (2024-02-16T03:30:27Z) - Elastic Multi-Gradient Descent for Parallel Continual Learning [28.749215705746135]
We study the novel paradigm of Parallel Continual Learning (PCL) in dynamic multi-task scenarios.
PCL presents challenges due to the training of an unspecified number of tasks with varying learning progress.
We propose a memory editing mechanism guided by the gradient computed using EMGD to balance the training between old and new tasks.
arXiv Detail & Related papers (2024-01-02T06:26:25Z) - On Task Performance and Model Calibration with Supervised and
Self-Ensembled In-Context Learning [71.44986275228747]
In-context learning (ICL) has become an efficient approach propelled by the recent advancements in large language models (LLMs)
However, both paradigms are prone to suffer from the critical problem of overconfidence (i.e., miscalibration)
arXiv Detail & Related papers (2023-12-21T11:55:10Z) - Incorporating Neuro-Inspired Adaptability for Continual Learning in
Artificial Intelligence [59.11038175596807]
Continual learning aims to empower artificial intelligence with strong adaptability to the real world.
Existing advances mainly focus on preserving memory stability to overcome catastrophic forgetting.
We propose a generic approach that appropriately attenuates old memories in parameter distributions to improve learning plasticity.
arXiv Detail & Related papers (2023-08-29T02:43:58Z) - Online Continual Learning via the Meta-learning Update with Multi-scale
Knowledge Distillation and Data Augmentation [4.109784267309124]
Continual learning aims to rapidly and continually learn the current task from a sequence of tasks.
One common limitation of this method is the data imbalance between the previous and current tasks.
We propose a novel framework called Meta-learning update via Multi-scale Knowledge Distillation and Data Augmentation.
arXiv Detail & Related papers (2022-09-12T10:03:53Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.