CURLoRA: Stable LLM Continual Fine-Tuning and Catastrophic Forgetting Mitigation

• URL: http://arxiv.org/abs/2408.14572v1
• Date: Mon, 26 Aug 2024 18:42:59 GMT
• Title: CURLoRA: Stable LLM Continual Fine-Tuning and Catastrophic Forgetting Mitigation
• Authors: Muhammad Fawi,
• Abstract summary: CURLoRA is a novel approach to fine-tuning large language models. It mitigates catastrophic forgetting and reduces the number of trainable parameters. It maintains model stability and performance across tasks while significantly reducing the number of trainable parameters.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper introduces CURLoRA, a novel approach to fine-tuning large language models (LLMs) that leverages CUR matrix decomposition in the context of Low-Rank Adaptation (LoRA). Our method addresses two critical challenges in LLM fine-tuning: mitigating catastrophic forgetting during continual learning and reducing the number of trainable parameters. We propose a unique modification to the CUR decomposition process, utilizing inverted probabilities for column and row selection which acts as an implicit regularization, and initializing the $U$ matrix as a zero matrix, and only fine-tuning it. We demonstrate through experiments on multiple datasets that CURLoRA outperforms standard LoRA in mitigating catastrophic forgetting. It maintains model stability and performance across tasks while significantly reducing the number of trainable parameters. Our results show that CURLoRA achieves very good and stable task accuracy while maintaining base model's perplexity scores fixed compared to LoRA upon continual fine-tuning, particularly in scenarios with limited data.

Related papers

• LoRA Done RITE: Robust Invariant Transformation Equilibration for LoRA Optimization [78.93425154518705]
  Low-rank adaption (LoRA) is a widely used parameter-efficient finetuning method for LLM that reduces memory requirements. This paper introduces LoRA-RITE, a novel adaptive matrix preconditioning method for LoRA optimization.
  arXiv  Detail & Related papers  (2024-10-27T22:57:12Z)
• Controlled Low-Rank Adaptation with Subspace Regularization for Continued Training on Large Language Models [13.56631686493347]
  Large language models (LLMs) exhibit remarkable capabilities in natural language processing but face catastrophic forgetting when learning new tasks. We propose Controlled LoRA (CLoRA), a subspace regularization method on LoRA structure.
  arXiv  Detail & Related papers  (2024-10-22T08:27:23Z)
• LoKO: Low-Rank Kalman Optimizer for Online Fine-Tuning of Large Models [21.889177019111525]
  Training large models with millions or even billions of parameters from scratch incurs substantial computational costs. We use Low-Rank Adaptation (LoRA) to adapt only a reduced number of parameters to specific tasks with gradient-baseds. We propose robust approaches that work well across a vast range of well-established computer vision and language models.
  arXiv  Detail & Related papers  (2024-10-15T12:41:31Z)
• Is Parameter Collision Hindering Continual Learning in LLMs? [50.57658782050275]
  Large Language Models (LLMs) often suffer from catastrophic forgetting when learning multiple tasks sequentially. We show that building non-collision parameters is a more critical interdependence factor in addressing CL challenges. We propose Non-collision Low-Rank Adaptation (N-LoRA), a simple yet effective approach leveraging low collision rates to enhance CL in LLMs.
  arXiv  Detail & Related papers  (2024-10-14T05:54:11Z)
• Randomized Asymmetric Chain of LoRA: The First Meaningful Theoretical Framework for Low-Rank Adaptation [58.288682735160585]
  Low-Rank Adaptation (LoRA) is a popular technique for finetuning models. LoRA often under performs when compared to full- parameter fine-tuning. We present a framework that rigorously analyzes the adaptation rates of LoRA methods.
  arXiv  Detail & Related papers  (2024-10-10T18:51:53Z)
• BiLoRA: A Bi-level Optimization Framework for Overfitting-Resilient Low-Rank Adaptation of Large Pre-trained Models [34.1111413429869]
  BiLoRA is an overfitting-alleviating fine-tuning approach based on bi-level optimization (BLO) tested on ten datasets covering natural language understanding and generation tasks.
  arXiv  Detail & Related papers  (2024-03-19T14:11:20Z)
• Towards Robust Model-Based Reinforcement Learning Against Adversarial Corruption [60.958746600254884]
  This study tackles the challenges of adversarial corruption in model-based reinforcement learning (RL) We introduce an algorithm called corruption-robust optimistic MLE (CR-OMLE), which leverages total-variation (TV)-based information ratios as uncertainty weights for MLE. We extend our weighting technique to the offline setting, and propose an algorithm named corruption-robust pessimistic MLE (CR-PMLE)
  arXiv  Detail & Related papers  (2024-02-14T07:27:30Z)
• Chain of LoRA: Efficient Fine-tuning of Language Models via Residual Learning [31.036465632204663]
  We introduce Chain of LoRA, an iterative optimization framework inspired by the Frank-Wolfe algorithm. We demonstrate that COLA can consistently outperform LoRA without additional computational or memory costs.
  arXiv  Detail & Related papers  (2024-01-08T14:26:49Z)
• On Continual Model Refinement in Out-of-Distribution Data Streams [64.62569873799096]
  Real-world natural language processing (NLP) models need to be continually updated to fix the prediction errors in out-of-distribution (OOD) data streams. Existing continual learning (CL) problem setups cannot cover such a realistic and complex scenario. We propose a new CL problem formulation dubbed continual model refinement (CMR)
  arXiv  Detail & Related papers  (2022-05-04T11:54:44Z)
• Solving weakly supervised regression problem using low-rank manifold regularization [77.34726150561087]
  We solve a weakly supervised regression problem. Under "weakly" we understand that for some training points the labels are known, for some unknown, and for others uncertain due to the presence of random noise or other reasons such as lack of resources. In the numerical section, we applied the suggested method to artificial and real datasets using Monte-Carlo modeling.
  arXiv  Detail & Related papers  (2021-04-13T23:21:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.