A Comprehensive Survey of Continual Learning: Theory, Method and Application

• URL: http://arxiv.org/abs/2302.00487v3
• Date: Tue, 6 Feb 2024 09:12:09 GMT
• Title: A Comprehensive Survey of Continual Learning: Theory, Method and Application
• Authors: Liyuan Wang, Xingxing Zhang, Hang Su, Jun Zhu
• Abstract summary: We present a comprehensive survey of continual learning, seeking to bridge the basic settings, theoretical foundations, representative methods, and practical applications. We summarize the general objectives of continual learning as ensuring a proper stability-plasticity trade-off and an adequate intra/inter-task generalizability in the context of resource efficiency.
• Score: 64.23253420555989
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: To cope with real-world dynamics, an intelligent system needs to incrementally acquire, update, accumulate, and exploit knowledge throughout its lifetime. This ability, known as continual learning, provides a foundation for AI systems to develop themselves adaptively. In a general sense, continual learning is explicitly limited by catastrophic forgetting, where learning a new task usually results in a dramatic performance degradation of the old tasks. Beyond this, increasingly numerous advances have emerged in recent years that largely extend the understanding and application of continual learning. The growing and widespread interest in this direction demonstrates its realistic significance as well as complexity. In this work, we present a comprehensive survey of continual learning, seeking to bridge the basic settings, theoretical foundations, representative methods, and practical applications. Based on existing theoretical and empirical results, we summarize the general objectives of continual learning as ensuring a proper stability-plasticity trade-off and an adequate intra/inter-task generalizability in the context of resource efficiency. Then we provide a state-of-the-art and elaborated taxonomy, extensively analyzing how representative methods address continual learning, and how they are adapted to particular challenges in realistic applications. Through an in-depth discussion of promising directions, we believe that such a holistic perspective can greatly facilitate subsequent exploration in this field and beyond.

Related papers

• Towards Incremental Learning in Large Language Models: A Critical Review [0.0]
  This review provides a comprehensive analysis of incremental learning in Large Language Models. It synthesizes the state-of-the-art incremental learning paradigms, including continual learning, meta-learning, parameter-efficient learning, and mixture-of-experts learning. An important finding is that many of these approaches do not update the core model, and none of them update incrementally in real-time.
  arXiv  Detail & Related papers  (2024-04-28T20:44:53Z)
• Open-world Machine Learning: A Review and New Outlooks [83.6401132743407]
  This paper aims to provide a comprehensive introduction to the emerging open-world machine learning paradigm. It aims to help researchers build more powerful AI systems in their respective fields, and to promote the development of artificial general intelligence.
  arXiv  Detail & Related papers  (2024-03-04T06:25:26Z)
• Evaluating and Improving Continual Learning in Spoken Language Understanding [58.723320551761525]
  We propose an evaluation methodology that provides a unified evaluation on stability, plasticity, and generalizability in continual learning. By employing the proposed metric, we demonstrate how introducing various knowledge distillations can improve different aspects of these three properties of the SLU model.
  arXiv  Detail & Related papers  (2024-02-16T03:30:27Z)
• A Definition of Continual Reinforcement Learning [69.56273766737527]
  In a standard view of the reinforcement learning problem, an agent's goal is to efficiently identify a policy that maximizes long-term reward. Continual reinforcement learning refers to the setting in which the best agents never stop learning. We formalize the notion of agents that "never stop learning" through a new mathematical language for analyzing and cataloging agents.
  arXiv  Detail & Related papers  (2023-07-20T17:28:01Z)
• The Ideal Continual Learner: An Agent That Never Forgets [11.172382217477129]
  The goal of continual learning is to find a model that solves multiple learning tasks which are presented sequentially to the learner. A key challenge in this setting is that the learner may forget how to solve a previous task when learning a new task, a phenomenon known as catastrophic forgetting. This paper proposes a new continual learning framework called Ideal Continual Learner (ICL) which is guaranteed to avoid catastrophic forgetting by construction.
  arXiv  Detail & Related papers  (2023-04-29T18:06:14Z)
• Unveiling the Tapestry: the Interplay of Generalization and Forgetting in Continual Learning [18.61040106667249]
  In AI, generalization refers to a model's ability to perform well on out-of-distribution data related to a given task, beyond the data it was trained on. Continual learning methods often include mechanisms to mitigate catastrophic forgetting, ensuring that knowledge from earlier tasks is retained. We introduce a simple and effective technique known as Shape-Texture Consistency Regularization (STCR), which caters to continual learning.
  arXiv  Detail & Related papers  (2022-11-21T04:36:24Z)
• Transferability in Deep Learning: A Survey [80.67296873915176]
  The ability to acquire and reuse knowledge is known as transferability in deep learning. We present this survey to connect different isolated areas in deep learning with their relation to transferability. We implement a benchmark and an open-source library, enabling a fair evaluation of deep learning methods in terms of transferability.
  arXiv  Detail & Related papers  (2022-01-15T15:03:17Z)
• Towards a theory of out-of-distribution learning [23.878004729029644]
  We propose a chronological approach to defining different learning tasks using the provably approximately correct (PAC) learning framework. We will start with in-distribution learning and progress to recently proposed lifelong or continual learning. Our hope is that this work will inspire a universally agreed-upon approach to quantifying different types of learning.
  arXiv  Detail & Related papers  (2021-09-29T15:35:16Z)
• Importance Weighted Policy Learning and Adaptation [89.46467771037054]
  We study a complementary approach which is conceptually simple, general, modular and built on top of recent improvements in off-policy learning. The framework is inspired by ideas from the probabilistic inference literature and combines robust off-policy learning with a behavior prior. Our approach achieves competitive adaptation performance on hold-out tasks compared to meta reinforcement learning baselines and can scale to complex sparse-reward scenarios.
  arXiv  Detail & Related papers  (2020-09-10T14:16:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.