Continual Learning for Large Language Models: A Survey

• URL: http://arxiv.org/abs/2402.01364v2
• Date: Wed, 7 Feb 2024 07:14:39 GMT
• Title: Continual Learning for Large Language Models: A Survey
• Authors: Tongtong Wu, Linhao Luo, Yuan-Fang Li, Shirui Pan, Thuy-Trang Vu, Gholamreza Haffari
• Abstract summary: Large language models (LLMs) are not amenable to frequent re-training, due to high training costs arising from their massive scale. This paper surveys recent works on continual learning for LLMs.
• Score: 95.79977915131145
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) are not amenable to frequent re-training, due to high training costs arising from their massive scale. However, updates are necessary to endow LLMs with new skills and keep them up-to-date with rapidly evolving human knowledge. This paper surveys recent works on continual learning for LLMs. Due to the unique nature of LLMs, we catalog continue learning techniques in a novel multi-staged categorization scheme, involving continual pretraining, instruction tuning, and alignment. We contrast continual learning for LLMs with simpler adaptation methods used in smaller models, as well as with other enhancement strategies like retrieval-augmented generation and model editing. Moreover, informed by a discussion of benchmarks and evaluation, we identify several challenges and future work directions for this crucial task.

Related papers

• Recent Advances of Foundation Language Models-based Continual Learning: A Survey [31.171203978742447]
  Foundation language models (LMs) have marked significant achievements in the domains of natural language processing (NLP) and computer vision (CV) However, they can not emulate human-like continuous learning due to catastrophic forgetting. Various continual learning (CL)-based methodologies have been developed to refine LMs, enabling them to adapt to new tasks without forgetting previous knowledge.
  arXiv  Detail & Related papers  (2024-05-28T23:32:46Z)
• When Life gives you LLMs, make LLM-ADE: Large Language Models with Adaptive Data Engineering [0.0]
  LLM-ADE is a methodology for continued pre-training of large language models. It addresses the challenges of catastrophic forgetting and double descent. It enhances model adaptability to new data while preserving previously acquired knowledge.
  arXiv  Detail & Related papers  (2024-04-19T17:43:26Z)
• Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMs [60.40396361115776]
  This paper introduces a novel collaborative approach, namely SlimPLM, that detects missing knowledge in large language models (LLMs) with a slim proxy model. We employ a proxy model which has far fewer parameters, and take its answers as answers. Heuristic answers are then utilized to predict the knowledge required to answer the user question, as well as the known and unknown knowledge within the LLM.
  arXiv  Detail & Related papers  (2024-02-19T11:11:08Z)
• Large Language Models: A Survey [69.72787936480394]
  Large Language Models (LLMs) have drawn a lot of attention due to their strong performance on a wide range of natural language tasks. LLMs' ability of general-purpose language understanding and generation is acquired by training billions of model's parameters on massive amounts of text data.
  arXiv  Detail & Related papers  (2024-02-09T05:37:09Z)
• Supervised Knowledge Makes Large Language Models Better In-context Learners [94.89301696512776]
  Large Language Models (LLMs) exhibit emerging in-context learning abilities through prompt engineering. The challenge of improving the generalizability and factuality of LLMs in natural language understanding and question answering remains under-explored. We propose a framework that enhances the reliability of LLMs as it: 1) generalizes out-of-distribution data, 2) elucidates how LLMs benefit from discriminative models, and 3) minimizes hallucinations in generative tasks.
  arXiv  Detail & Related papers  (2023-12-26T07:24:46Z)
• TRACE: A Comprehensive Benchmark for Continual Learning in Large Language Models [52.734140807634624]
  Aligned large language models (LLMs) demonstrate exceptional capabilities in task-solving, following instructions, and ensuring safety. Existing continual learning benchmarks lack sufficient challenge for leading aligned LLMs. We introduce TRACE, a novel benchmark designed to evaluate continual learning in LLMs.
  arXiv  Detail & Related papers  (2023-10-10T16:38:49Z)
• Rethinking Learning Rate Tuning in the Era of Large Language Models [11.87985768634266]
  Large Language Models (LLMs) represent the recent success of deep learning in achieving remarkable human-like predictive performance. It has become a mainstream strategy to leverage fine-tuning to adapt LLMs for various real-world applications. Existing learning rate policies are primarily designed for training traditional deep neural networks (DNNs)
  arXiv  Detail & Related papers  (2023-09-16T03:37:00Z)
• Scaling Sentence Embeddings with Large Language Models [43.19994568210206]
  In this work, we propose an in-context learning-based method aimed at improving sentence embeddings performance. Our approach involves adapting the previous prompt-based representation method for autoregressive models. By scaling model size, we find scaling to more than tens of billion parameters harms the performance on semantic textual similarity tasks.
  arXiv  Detail & Related papers  (2023-07-31T13:26:03Z)
• Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning [74.07455280246212]
  Continual learning studies agents that learn from streams of tasks without forgetting previous ones while adapting to new ones. We show that current continual learning, meta-learning, meta-continual learning, and continual-meta learning techniques fail in this new scenario. We propose Continual-MAML, an online extension of the popular MAML algorithm as a strong baseline for this scenario.
  arXiv  Detail & Related papers  (2020-03-12T15:47:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.