The Bayesian Approach to Continual Learning: An Overview

• URL: http://arxiv.org/abs/2507.08922v1
• Date: Fri, 11 Jul 2025 17:16:13 GMT
• Title: The Bayesian Approach to Continual Learning: An Overview
• Authors: Tameem Adel,
• Abstract summary: Continual learning is an online paradigm where a learner continually accumulates knowledge from different tasks.<n>Continual need to update the learner with data arriving sequentially strikes inherent congruence between continual learning and Bayesian inference.<n>This survey inspects different settings of Bayesian continual learning, namely task-incremental learning and class-incremental learning.
• Score: 4.550647089601897
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Continual learning is an online paradigm where a learner continually accumulates knowledge from different tasks encountered over sequential time steps. Importantly, the learner is required to extend and update its knowledge without forgetting about the learning experience acquired from the past, and while avoiding the need to retrain from scratch. Given its sequential nature and its resemblance to the way humans think, continual learning offers an opportunity to address several challenges which currently stand in the way of widening the range of applicability of deep models to further real-world problems. The continual need to update the learner with data arriving sequentially strikes inherent congruence between continual learning and Bayesian inference which provides a principal platform to keep updating the prior beliefs of a model given new data, without completely forgetting the knowledge acquired from the old data. This survey inspects different settings of Bayesian continual learning, namely task-incremental learning and class-incremental learning. We begin by discussing definitions of continual learning along with its Bayesian setting, as well as the links with related fields, such as domain adaptation, transfer learning and meta-learning. Afterwards, we introduce a taxonomy offering a comprehensive categorization of algorithms belonging to the Bayesian continual learning paradigm. Meanwhile, we analyze the state-of-the-art while zooming in on some of the most prominent Bayesian continual learning algorithms to date. Furthermore, we shed some light on links between continual learning and developmental psychology, and correspondingly introduce analogies between both fields. We follow that with a discussion of current challenges, and finally conclude with potential areas for future research on Bayesian continual learning.

Related papers

• The Future of Continual Learning in the Era of Foundation Models: Three Key Directions [3.805777835466912]
  We argue that continual learning remains essential for three key reasons.<n>We argue it is continual compositionality that will mark the rebirth of continual learning.<n>The future of AI will not be defined by a single static model but by an ecosystem of continually evolving and interacting models.
  arXiv  Detail & Related papers  (2025-06-03T19:06:41Z)
• Mathematics of Continual Learning [45.73704932193178]
  Continual learning aims to solve multiple tasks presented sequentially to the learner without forgetting previously learned tasks.<n> adaptive filtering is a classic subject in signal processing with a rich history of mathematically principled methods.
  arXiv  Detail & Related papers  (2025-04-24T22:23:41Z)
• Continual Learning: Applications and the Road Forward [119.03464063873407]
  Continual learning aims to allow machine learning models to continuously learn on new data, by accumulating knowledge without forgetting what was learned in the past. This work is the result of the many discussions the authors had at the Dagstuhl seminar on Deep Continual Learning, in March 2023.
  arXiv  Detail & Related papers  (2023-11-20T16:40:29Z)
• When Meta-Learning Meets Online and Continual Learning: A Survey [39.53836535326121]
  meta-learning is a data-driven approach to optimize the learning algorithm. Continual learning and online learning, both of which involve incrementally updating a model with streaming data. This paper organizes various problem settings using consistent terminology and formal descriptions.
  arXiv  Detail & Related papers  (2023-11-09T09:49:50Z)
• COOLer: Class-Incremental Learning for Appearance-Based Multiple Object Tracking [32.47215340215641]
  This paper extends the scope of continual learning research to class-incremental learning for multiple object tracking (MOT) Previous solutions for continual learning of object detectors do not address the data association stage of appearance-based trackers. We introduce COOLer, a COntrastive- and cOntinual-Learning-based tracker, which incrementally learns to track new categories while preserving past knowledge.
  arXiv  Detail & Related papers  (2023-10-04T17:49:48Z)
• On Generalizing Beyond Domains in Cross-Domain Continual Learning [91.56748415975683]
  Deep neural networks often suffer from catastrophic forgetting of previously learned knowledge after learning a new task. Our proposed approach learns new tasks under domain shift with accuracy boosts up to 10% on challenging datasets such as DomainNet and OfficeHome.
  arXiv  Detail & Related papers  (2022-03-08T09:57:48Z)
• Transferability in Deep Learning: A Survey [80.67296873915176]
  The ability to acquire and reuse knowledge is known as transferability in deep learning. We present this survey to connect different isolated areas in deep learning with their relation to transferability. We implement a benchmark and an open-source library, enabling a fair evaluation of deep learning methods in terms of transferability.
  arXiv  Detail & Related papers  (2022-01-15T15:03:17Z)
• Online Continual Learning with Natural Distribution Shifts: An Empirical Study with Visual Data [101.6195176510611]
  "Online" continual learning enables evaluating both information retention and online learning efficacy. In online continual learning, each incoming small batch of data is first used for testing and then added to the training set, making the problem truly online. We introduce a new benchmark for online continual visual learning that exhibits large scale and natural distribution shifts.
  arXiv  Detail & Related papers  (2021-08-20T06:17:20Z)
• Transfer Learning in Deep Reinforcement Learning: A Survey [64.36174156782333]
  Reinforcement learning is a learning paradigm for solving sequential decision-making problems. Recent years have witnessed remarkable progress in reinforcement learning upon the fast development of deep neural networks. transfer learning has arisen to tackle various challenges faced by reinforcement learning.
  arXiv  Detail & Related papers  (2020-09-16T18:38:54Z)
• Bilevel Continual Learning [76.50127663309604]
  We present a novel framework of continual learning named "Bilevel Continual Learning" (BCL) Our experiments on continual learning benchmarks demonstrate the efficacy of the proposed BCL compared to many state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-07-30T16:00:23Z)
• Meta-Learning in Neural Networks: A Survey [4.588028371034406]
  This survey describes the contemporary meta-learning landscape. We first discuss definitions of meta-learning and position it with respect to related fields. We then propose a new taxonomy that provides a more comprehensive breakdown of the space of meta-learning methods.
  arXiv  Detail & Related papers  (2020-04-11T16:34:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.