Continual Learning through Networks Splitting and Merging with Dreaming-Meta-Weighted Model Fusion

• URL: http://arxiv.org/abs/2312.07082v1
• Date: Tue, 12 Dec 2023 09:02:56 GMT
• Title: Continual Learning through Networks Splitting and Merging with Dreaming-Meta-Weighted Model Fusion
• Authors: Yi Sun, Xin Xu, Jian Li, Guanglei Xie, Yifei Shi, Qiang Fang
• Abstract summary: It's challenging to balance the networks stability and plasticity in continual learning scenarios. We propose Split2MetaFusion which can achieve better trade-off by employing a two-stage strategy.
• Score: 20.74264925323055
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: It's challenging to balance the networks stability and plasticity in continual learning scenarios, considering stability suffers from the update of model and plasticity benefits from it. Existing works usually focus more on the stability and restrict the learning plasticity of later tasks to avoid catastrophic forgetting of learned knowledge. Differently, we propose a continual learning method named Split2MetaFusion which can achieve better trade-off by employing a two-stage strategy: splitting and meta-weighted fusion. In this strategy, a slow model with better stability, and a fast model with better plasticity are learned sequentially at the splitting stage. Then stability and plasticity are both kept by fusing the two models in an adaptive manner. Towards this end, we design an optimizer named Task-Preferred Null Space Projector(TPNSP) to the slow learning process for narrowing the fusion gap. To achieve better model fusion, we further design a Dreaming-Meta-Weighted fusion policy for better maintaining the old and new knowledge simultaneously, which doesn't require to use the previous datasets. Experimental results and analysis reported in this work demonstrate the superiority of the proposed method for maintaining networks stability and keeping its plasticity. Our code will be released.

Related papers

• Action Flow Matching for Continual Robot Learning [57.698553219660376]
  Continual learning in robotics seeks systems that can constantly adapt to changing environments and tasks. We introduce a generative framework leveraging flow matching for online robot dynamics model alignment. We find that by transforming the actions themselves rather than exploring with a misaligned model, the robot collects informative data more efficiently.
  arXiv  Detail & Related papers  (2025-04-25T16:26:15Z)
• BECAME: BayEsian Continual Learning with Adaptive Model MErging [21.642774366793997]
  We introduce a two-stage framework named BECAME, which synergizes the expertise of gradient projection and adaptive merging. Our approach outperforms state-of-the-art CL methods and existing merging strategies.
  arXiv  Detail & Related papers  (2025-04-03T15:07:28Z)
• Neural Networks Remember More: The Power of Parameter Isolation and Combination [3.2430260063115233]
  Catastrophic forgetting is a pervasive issue for pre-trained language models. Key to solving this problem is to find a trade-off between the plasticity and stability of the model. We propose a novel method to achieve a balance between model stability and plasticity.
  arXiv  Detail & Related papers  (2025-02-16T02:58:57Z)
• Merging Models on the Fly Without Retraining: A Sequential Approach to Scalable Continual Model Merging [75.93960998357812]
  Deep model merging represents an emerging research direction that combines multiple fine-tuned models to harness their capabilities across different tasks and domains. Current model merging techniques focus on merging all available models simultaneously, with weight matrices-based methods being the predominant approaches. We propose a training-free projection-based continual merging method that processes models sequentially.
  arXiv  Detail & Related papers  (2025-01-16T13:17:24Z)
• Exploring Model Kinship for Merging Large Language Models [52.01652098827454]
  We introduce model kinship, the degree of similarity or relatedness between Large Language Models. We find that there is a certain relationship between model kinship and the performance gains after model merging. We propose a new model merging strategy: Top-k Greedy Merging with Model Kinship, which can yield better performance on benchmark datasets.
  arXiv  Detail & Related papers  (2024-10-16T14:29:29Z)
• LLaCA: Multimodal Large Language Continual Assistant [59.585544987096974]
  Multimodal Continual Instruction Tuning (MCIT) is adopted to continually instruct MLLMs to follow human intent in sequential datasets. Existing gradient update would heavily destroy the tuning performance on previous datasets. We propose a method called Multimodal Large Language Continual Assistant (LLaCA) to address the challenge.
  arXiv  Detail & Related papers  (2024-10-08T11:24:59Z)
• Model Merging and Safety Alignment: One Bad Model Spoils the Bunch [70.614652904151]
  Merging Large Language Models (LLMs) is a cost-effective technique for combining multiple expert LLMs into a single versatile model. Current approaches often overlook the importance of safety alignment during merging, leading to highly misaligned models. We evaluate several popular model merging techniques, demonstrating that existing methods do not only transfer domain expertise but also propagate misalignment.
  arXiv  Detail & Related papers  (2024-06-20T17:59:58Z)
• Towards Stable Machine Learning Model Retraining via Slowly Varying Sequences [6.067007470552307]
  We propose a methodology for finding sequences of machine learning models that are stable across retraining iterations. We develop a mixed-integer optimization formulation that is guaranteed to recover optimal models. Our method shows stronger stability than greedily trained models with a small, controllable sacrifice in predictive power.
  arXiv  Detail & Related papers  (2024-03-28T22:45:38Z)
• Branch-Tuning: Balancing Stability and Plasticity for Continual Self-Supervised Learning [33.560003528712414]
  Self-supervised learning (SSL) has emerged as an effective paradigm for deriving general representations from vast amounts of unlabeled data. This poses a challenge in striking a balance between stability and plasticity when adapting to new information. We propose Branch-tuning, an efficient and straightforward method that achieves a balance between stability and plasticity in continual SSL.
  arXiv  Detail & Related papers  (2024-03-27T05:38:48Z)
• New Insights for the Stability-Plasticity Dilemma in Online Continual Learning [21.664470275289407]
  We propose an online continual learning framework named multi-scale feature adaptation network (MuFAN) MuFAN outperforms other state-of-the-art continual learning methods on the SVHN, CIFAR100, miniImageNet, and CORe50 datasets.
  arXiv  Detail & Related papers  (2023-02-17T07:43:59Z)
• PlaStIL: Plastic and Stable Memory-Free Class-Incremental Learning [49.0417577439298]
  Plasticity and stability are needed in class-incremental learning in order to learn from new data while preserving past knowledge. We propose a method which has similar number of parameters but distributes them differently to find a better balance between plasticity and stability.
  arXiv  Detail & Related papers  (2022-09-14T12:53:00Z)
• Balancing Stability and Plasticity through Advanced Null Space in Continual Learning [77.94570903726856]
  We propose a new continual learning approach, Advanced Null Space (AdNS), to balance the stability and plasticity without storing any old data of previous tasks. We also present a simple but effective method, intra-task distillation, to improve the performance of the current task. Experimental results show that the proposed method can achieve better performance compared to state-of-the-art continual learning approaches.
  arXiv  Detail & Related papers  (2022-07-25T11:04:22Z)
• FOSTER: Feature Boosting and Compression for Class-Incremental Learning [52.603520403933985]
  Deep neural networks suffer from catastrophic forgetting when learning new categories. We propose a novel two-stage learning paradigm FOSTER, empowering the model to learn new categories adaptively.
  arXiv  Detail & Related papers  (2022-04-10T11:38:33Z)
• Towards Better Plasticity-Stability Trade-off in Incremental Learning: A simple Linear Connector [8.13916229438606]
  Plasticity-stability dilemma is a main problem for incremental learning. We show that a simple averaging of two independently optimized optima of networks, null-space projection for past tasks and simple SGD for the current task, can attain a meaningful balance between preserving already learned knowledge and granting sufficient flexibility for learning a new task.
  arXiv  Detail & Related papers  (2021-10-15T07:37:20Z)
• Enabling Continual Learning with Differentiable Hebbian Plasticity [18.12749708143404]
  Continual learning is the problem of sequentially learning new tasks or knowledge while protecting previously acquired knowledge. catastrophic forgetting poses a grand challenge for neural networks performing such learning process. We propose a Differentiable Hebbian Consolidation model which is composed of a Differentiable Hebbian Plasticity.
  arXiv  Detail & Related papers  (2020-06-30T06:42:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.