Large Language Model Can Continue Evolving From Mistakes

• URL: http://arxiv.org/abs/2404.08707v4
• Date: Mon, 17 Jun 2024 11:32:29 GMT
• Title: Large Language Model Can Continue Evolving From Mistakes
• Authors: Haokun Zhao, Haixia Han, Jie Shi, Chengyu Du, Jiaqing Liang, Yanghua Xiao,
• Abstract summary: Continual learning is crucial for keeping Large Language Models (LLMs) up-to-date and addressing their shortcomings. We propose the Continue Evolving from Mistakes (CEM) method, aiming to provide a data-efficient approach for collecting continual CPT data. Experiments demonstrate the efficacy of the CEM method, achieving up to a 17% improvement in accuracy in the best case.
• Score: 36.14056870453356
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As world knowledge evolves and new task paradigms emerge, Continual Learning (CL) is crucial for keeping Large Language Models (LLMs) up-to-date and addressing their shortcomings. In practical applications, LLMs often require both continual instruction tuning (CIT) and continual pre-training (CPT) to adapt to new task paradigms and acquire necessary knowledge for task-solving. However, it remains challenging to collect CPT data that addresses the knowledge deficiencies in models while maintaining adequate volume, and improving the efficiency of utilizing this data also presents significant difficulties. Inspired by the 'summarizing mistakes' learning skill, we propose the Continue Evolving from Mistakes (CEM) method, aiming to provide a data-efficient approach for collecting CPT data and continually improving LLMs' performance through iterative evaluation and supplementation with mistake-relevant knowledge. To efficiently utilize these CPT data and mitigate forgetting, we design a novel CL training set construction paradigm that integrates parallel CIT and CPT data. Extensive experiments demonstrate the efficacy of the CEM method, achieving up to a 17% improvement in accuracy in the best case. Furthermore, additional experiments confirm the potential of combining CEM with catastrophic forgetting mitigation methods, enabling iterative and continual model evolution.

Related papers

• MoExtend: Tuning New Experts for Modality and Task Extension [61.29100693866109]
  MoExtend is an effective framework designed to streamline the modality adaptation and extension of Mixture-of-Experts (MoE) models. MoExtend seamlessly integrates new experts into pre-trained MoE models, endowing them with novel knowledge without the need to tune pretrained models.
  arXiv  Detail & Related papers  (2024-08-07T02:28:37Z)
• Learning to Learn without Forgetting using Attention [5.6739565497512405]
  Continual learning (CL) refers to the ability to continually learn over time by accommodating new knowledge while retaining previously learned experience. Current machine learning methods are highly prone to overwrite previously learned patterns and thus forget past experience. Since hand-crafting effective update mechanisms is difficult, we propose meta-learning a transformer-based to enhance CL.
  arXiv  Detail & Related papers  (2024-08-06T14:25:23Z)
• Learn while Unlearn: An Iterative Unlearning Framework for Generative Language Models [49.043599241803825]
  Iterative Contrastive Unlearning (ICU) framework consists of three core components. A Knowledge Unlearning Induction module removes specific knowledge through an unlearning loss. A Contrastive Learning Enhancement module to preserve the model's expressive capabilities against the pure unlearning goal. And an Iterative Unlearning Refinement module that dynamically assess the unlearning extent on specific data pieces and make iterative update.
  arXiv  Detail & Related papers  (2024-07-25T07:09:35Z)
• Learning to Unlearn for Robust Machine Unlearning [6.488418950340473]
  We introduce a novel Learning-to-Unlearn (LTU) framework to optimize the unlearning process. LTU includes a meta-optimization scheme that facilitates models to effectively preserve generalizable knowledge. We also introduce a Gradient Harmonization strategy to align the optimization trajectories for remembering and forgetting.
  arXiv  Detail & Related papers  (2024-07-15T07:36:00Z)
• DELTA: Decoupling Long-Tailed Online Continual Learning [7.507868991415516]
  Long-Tailed Online Continual Learning (LTOCL) aims to learn new tasks from sequentially arriving class-imbalanced data streams. We present DELTA, a decoupled learning approach designed to enhance learning representations. We demonstrate that DELTA improves the capacity for incremental learning, surpassing existing OCL methods.
  arXiv  Detail & Related papers  (2024-04-06T02:33:04Z)
• Continual Learning for Large Language Models: A Survey [95.79977915131145]
  Large language models (LLMs) are not amenable to frequent re-training, due to high training costs arising from their massive scale. This paper surveys recent works on continual learning for LLMs.
  arXiv  Detail & Related papers  (2024-02-02T12:34:09Z)
• Unlearn What You Want to Forget: Efficient Unlearning for LLMs [92.51670143929056]
  Large language models (LLMs) have achieved significant progress from pre-training on and memorizing a wide range of textual data. This process might suffer from privacy issues and violations of data protection regulations. We propose an efficient unlearning framework that could efficiently update LLMs without having to retrain the whole model after data removals.
  arXiv  Detail & Related papers  (2023-10-31T03:35:59Z)
• CLIP: Train Faster with Less Data [3.2575001434344286]
  Deep learning models require an enormous amount of data for training. Recently there is a shift in machine learning from model-centric to data-centric approaches. We propose CLIP i.e., Curriculum Learning with Iterative data Pruning.
  arXiv  Detail & Related papers  (2022-12-02T21:29:48Z)
• Bilevel Continual Learning [76.50127663309604]
  We present a novel framework of continual learning named "Bilevel Continual Learning" (BCL) Our experiments on continual learning benchmarks demonstrate the efficacy of the proposed BCL compared to many state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-07-30T16:00:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.