Class-Incremental Learning: A Survey

• URL: http://arxiv.org/abs/2302.03648v2
• Date: Mon, 15 Jul 2024 13:35:33 GMT
• Title: Class-Incremental Learning: A Survey
• Authors: Da-Wei Zhou, Qi-Wei Wang, Zhi-Hong Qi, Han-Jia Ye, De-Chuan Zhan, Ziwei Liu,
• Abstract summary: Class-Incremental Learning (CIL) enables the learner to incorporate the knowledge of new classes incrementally. CIL tends to catastrophically forget the characteristics of former ones, and its performance drastically degrades. We provide a rigorous and unified evaluation of 17 methods in benchmark image classification tasks to find out the characteristics of different algorithms.
• Score: 84.30083092434938
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep models, e.g., CNNs and Vision Transformers, have achieved impressive achievements in many vision tasks in the closed world. However, novel classes emerge from time to time in our ever-changing world, requiring a learning system to acquire new knowledge continually. Class-Incremental Learning (CIL) enables the learner to incorporate the knowledge of new classes incrementally and build a universal classifier among all seen classes. Correspondingly, when directly training the model with new class instances, a fatal problem occurs -- the model tends to catastrophically forget the characteristics of former ones, and its performance drastically degrades. There have been numerous efforts to tackle catastrophic forgetting in the machine learning community. In this paper, we survey comprehensively recent advances in class-incremental learning and summarize these methods from several aspects. We also provide a rigorous and unified evaluation of 17 methods in benchmark image classification tasks to find out the characteristics of different algorithms empirically. Furthermore, we notice that the current comparison protocol ignores the influence of memory budget in model storage, which may result in unfair comparison and biased results. Hence, we advocate fair comparison by aligning the memory budget in evaluation, as well as several memory-agnostic performance measures. The source code is available at https://github.com/zhoudw-zdw/CIL_Survey/

Related papers

• Happy: A Debiased Learning Framework for Continual Generalized Category Discovery [54.54153155039062]
  This paper explores the underexplored task of Continual Generalized Category Discovery (C-GCD) C-GCD aims to incrementally discover new classes from unlabeled data while maintaining the ability to recognize previously learned classes. We introduce a debiased learning framework, namely Happy, characterized by Hardness-aware prototype sampling and soft entropy regularization.
  arXiv  Detail & Related papers  (2024-10-09T04:18:51Z)
• PASS++: A Dual Bias Reduction Framework for Non-Exemplar Class-Incremental Learning [49.240408681098906]
  Class-incremental learning (CIL) aims to recognize new classes incrementally while maintaining the discriminability of old classes. Most existing CIL methods are exemplar-based, i.e., storing a part of old data for retraining. We present a simple and novel dual bias reduction framework that employs self-supervised transformation (SST) in input space and prototype augmentation (protoAug) in deep feature space.
  arXiv  Detail & Related papers  (2024-07-19T05:03:16Z)
• Towards Non-Exemplar Semi-Supervised Class-Incremental Learning [33.560003528712414]
  Class-incremental learning aims to gradually recognize new classes while maintaining the discriminability of old ones. We propose a non-exemplar semi-supervised CIL framework with contrastive learning and semi-supervised incremental prototype classifier (Semi-IPC) Semi-IPC learns a prototype for each class with unsupervised regularization, enabling the model to incrementally learn from partially labeled new data.
  arXiv  Detail & Related papers  (2024-03-27T06:28:19Z)
• Few-Shot Class-Incremental Learning via Training-Free Prototype Calibration [67.69532794049445]
  We find a tendency for existing methods to misclassify the samples of new classes into base classes, which leads to the poor performance of new classes. We propose a simple yet effective Training-frEE calibratioN (TEEN) strategy to enhance the discriminability of new classes.
  arXiv  Detail & Related papers  (2023-12-08T18:24:08Z)
• Class Incremental Learning with Self-Supervised Pre-Training and Prototype Learning [21.901331484173944]
  We analyze the causes of catastrophic forgetting in class incremental learning. We propose a two-stage learning framework with a fixed encoder and an incrementally updated prototype classifier. Our method does not rely on preserved samples of old classes, is thus a non-exemplar based CIL method.
  arXiv  Detail & Related papers  (2023-08-04T14:20:42Z)
• Few-Shot Class-Incremental Learning by Sampling Multi-Phase Tasks [59.12108527904171]
  A model should recognize new classes and maintain discriminability over old classes. The task of recognizing few-shot new classes without forgetting old classes is called few-shot class-incremental learning (FSCIL) We propose a new paradigm for FSCIL based on meta-learning by LearnIng Multi-phase Incremental Tasks (LIMIT)
  arXiv  Detail & Related papers  (2022-03-31T13:46:41Z)
• Self-Supervised Class Incremental Learning [51.62542103481908]
  Existing Class Incremental Learning (CIL) methods are based on a supervised classification framework sensitive to data labels. When updating them based on the new class data, they suffer from catastrophic forgetting: the model cannot discern old class data clearly from the new. In this paper, we explore the performance of Self-Supervised representation learning in Class Incremental Learning (SSCIL) for the first time.
  arXiv  Detail & Related papers  (2021-11-18T06:58:19Z)
• ClaRe: Practical Class Incremental Learning By Remembering Previous Class Representations [9.530976792843495]
  Class Incremental Learning (CIL) tends to learn new concepts perfectly, but not at the expense of performance and accuracy for old data. ClaRe is an efficient solution for CIL by remembering the representations of learned classes in each increment. ClaRe has a better generalization than prior methods thanks to producing diverse instances from the distribution of previously learned classes.
  arXiv  Detail & Related papers  (2021-03-29T10:39:42Z)
• Class-incremental Learning with Rectified Feature-Graph Preservation [24.098892115785066]
  A central theme of this paper is to learn new classes that arrive in sequential phases over time. We propose a weighted-Euclidean regularization for old knowledge preservation. We show how it can work with binary cross-entropy to increase class separation for effective learning of new classes.
  arXiv  Detail & Related papers  (2020-12-15T07:26:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.