Continual Learning of Large Language Models: A Comprehensive Survey

• URL: http://arxiv.org/abs/2404.16789v2
• Date: Sun, 30 Jun 2024 02:19:00 GMT
• Title: Continual Learning of Large Language Models: A Comprehensive Survey
• Authors: Haizhou Shi, Zihao Xu, Hengyi Wang, Weiyi Qin, Wenyuan Wang, Yibin Wang, Zifeng Wang, Sayna Ebrahimi, Hao Wang,
• Abstract summary: Large language models (LLMs) trained on static, pre-collected, general datasets has sparked numerous research directions and applications. One such direction addresses the non-trivial challenge of integrating pre-trained LLMs into dynamic data distributions, task structures, and user preferences. While extensively studied in the continual learning (CL) community, it presents new manifestations in the realm of LLMs.
• Score: 18.546766135948154
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The recent success of large language models (LLMs) trained on static, pre-collected, general datasets has sparked numerous research directions and applications. One such direction addresses the non-trivial challenge of integrating pre-trained LLMs into dynamic data distributions, task structures, and user preferences. Pre-trained LLMs, when tailored for specific needs, often experience significant performance degradation in previous knowledge domains -- a phenomenon known as "catastrophic forgetting". While extensively studied in the continual learning (CL) community, it presents new manifestations in the realm of LLMs. In this survey, we provide a comprehensive overview of the current research progress on LLMs within the context of CL. This survey is structured into four main sections: we first describe an overview of continually learning LLMs, consisting of two directions of continuity: vertical continuity (or vertical continual learning), i.e., continual adaptation from general to specific capabilities, and horizontal continuity (or horizontal continual learning), i.e., continual adaptation across time and domains (Section 3). We then summarize three stages of learning LLMs in the context of modern CL: Continual Pre-Training (CPT), Domain-Adaptive Pre-training (DAP), and Continual Fine-Tuning (CFT) (Section 4). Then we provide an overview of evaluation protocols for continual learning with LLMs, along with the current available data sources (Section 5). Finally, we discuss intriguing questions pertaining to continual learning for LLMs (Section 6). The full list of papers examined in this survey is available at https://github.com/Wang-ML-Lab/llm-continual-learning-survey.

Related papers

• Beyond Binary: Towards Fine-Grained LLM-Generated Text Detection via Role Recognition and Involvement Measurement [51.601916604301685]
  Large language models (LLMs) generate content that can undermine trust in online discourse. Current methods often focus on binary classification, failing to address the complexities of real-world scenarios like human-AI collaboration. To move beyond binary classification and address these challenges, we propose a new paradigm for detecting LLM-generated content.
  arXiv  Detail & Related papers  (2024-10-18T08:14:10Z)
• Federated Large Language Models: Current Progress and Future Directions [63.68614548512534]
  This paper surveys Federated learning for LLMs (FedLLM), highlighting recent advances and future directions. We focus on two key aspects: fine-tuning and prompt learning in a federated setting, discussing existing work and associated research challenges.
  arXiv  Detail & Related papers  (2024-09-24T04:14:33Z)
• Towards Lifelong Learning of Large Language Models: A Survey [20.0936011355535]
  This survey delves into the sophisticated landscape of lifelong learning, categorizing strategies into two primary groups: Internal Knowledge and External Knowledge. This study aims to enhance the adaptability, reliability, and overall performance of large language models in real-world applications.
  arXiv  Detail & Related papers  (2024-06-10T15:46:25Z)
• Recent Advances of Foundation Language Models-based Continual Learning: A Survey [31.171203978742447]
  Foundation language models (LMs) have marked significant achievements in the domains of natural language processing (NLP) and computer vision (CV) However, they can not emulate human-like continuous learning due to catastrophic forgetting. Various continual learning (CL)-based methodologies have been developed to refine LMs, enabling them to adapt to new tasks without forgetting previous knowledge.
  arXiv  Detail & Related papers  (2024-05-28T23:32:46Z)
• Continual Learning for Large Language Models: A Survey [95.79977915131145]
  Large language models (LLMs) are not amenable to frequent re-training, due to high training costs arising from their massive scale. This paper surveys recent works on continual learning for LLMs.
  arXiv  Detail & Related papers  (2024-02-02T12:34:09Z)
• Examining Forgetting in Continual Pre-training of Aligned Large Language Models [66.62800021628276]
  We investigate the phenomenon of forgetting that occurs during continual pre-training on an existing fine-tuned LLM. Experiment results highlight the non-trivial challenge of addressing catastrophic forgetting during continual pre-training.
  arXiv  Detail & Related papers  (2024-01-06T05:34:09Z)
• Supervised Knowledge Makes Large Language Models Better In-context Learners [94.89301696512776]
  Large Language Models (LLMs) exhibit emerging in-context learning abilities through prompt engineering. The challenge of improving the generalizability and factuality of LLMs in natural language understanding and question answering remains under-explored. We propose a framework that enhances the reliability of LLMs as it: 1) generalizes out-of-distribution data, 2) elucidates how LLMs benefit from discriminative models, and 3) minimizes hallucinations in generative tasks.
  arXiv  Detail & Related papers  (2023-12-26T07:24:46Z)
• A Survey on Large Language Models for Software Engineering [15.468484685849983]
  Large Language Models (LLMs) are used to automate a broad range of Software Engineering (SE) tasks. This paper summarizes the current state-of-the-art research in the LLM-based SE community.
  arXiv  Detail & Related papers  (2023-12-23T11:09:40Z)
• TRACE: A Comprehensive Benchmark for Continual Learning in Large Language Models [52.734140807634624]
  Aligned large language models (LLMs) demonstrate exceptional capabilities in task-solving, following instructions, and ensuring safety. Existing continual learning benchmarks lack sufficient challenge for leading aligned LLMs. We introduce TRACE, a novel benchmark designed to evaluate continual learning in LLMs.
  arXiv  Detail & Related papers  (2023-10-10T16:38:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.