A Survey on Few-Shot Class-Incremental Learning

• URL: http://arxiv.org/abs/2304.08130v2
• Date: Tue, 24 Oct 2023 03:08:47 GMT
• Title: A Survey on Few-Shot Class-Incremental Learning
• Authors: Songsong Tian, Lusi Li, Weijun Li, Hang Ran, Xin Ning, Prayag Tiwari
• Abstract summary: Few-shot class-incremental learning (FSCIL) poses a significant challenge for deep neural networks to learn new tasks. This paper provides a comprehensive survey on FSCIL. FSCIL has achieved impressive achievements in various fields of computer vision.
• Score: 11.68962265057818
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large deep learning models are impressive, but they struggle when real-time data is not available. Few-shot class-incremental learning (FSCIL) poses a significant challenge for deep neural networks to learn new tasks from just a few labeled samples without forgetting the previously learned ones. This setup easily leads to catastrophic forgetting and overfitting problems, severely affecting model performance. Studying FSCIL helps overcome deep learning model limitations on data volume and acquisition time, while improving practicality and adaptability of machine learning models. This paper provides a comprehensive survey on FSCIL. Unlike previous surveys, we aim to synthesize few-shot learning and incremental learning, focusing on introducing FSCIL from two perspectives, while reviewing over 30 theoretical research studies and more than 20 applied research studies. From the theoretical perspective, we provide a novel categorization approach that divides the field into five subcategories, including traditional machine learning methods, meta-learning based methods, feature and feature space-based methods, replay-based methods, and dynamic network structure-based methods. We also evaluate the performance of recent theoretical research on benchmark datasets of FSCIL. From the application perspective, FSCIL has achieved impressive achievements in various fields of computer vision such as image classification, object detection, and image segmentation, as well as in natural language processing and graph. We summarize the important applications. Finally, we point out potential future research directions, including applications, problem setups, and theory development. Overall, this paper offers a comprehensive analysis of the latest advances in FSCIL from a methodological, performance, and application perspective.

Related papers

• Exploring the Precise Dynamics of Single-Layer GAN Models: Leveraging Multi-Feature Discriminators for High-Dimensional Subspace Learning [0.0]
  We study the training dynamics of a single-layer GAN model from the perspective of subspace learning. By bridging our analysis to the realm of subspace learning, we systematically compare the efficacy of GAN-based methods against conventional approaches.
  arXiv  Detail & Related papers  (2024-11-01T10:21:12Z)
• Deep Learning Through A Telescoping Lens: A Simple Model Provides Empirical Insights On Grokking, Gradient Boosting & Beyond [61.18736646013446]
  In pursuit of a deeper understanding of its surprising behaviors, we investigate the utility of a simple yet accurate model of a trained neural network. Across three case studies, we illustrate how it can be applied to derive new empirical insights on a diverse range of prominent phenomena.
  arXiv  Detail & Related papers  (2024-10-31T22:54:34Z)
• Recent Advances on Machine Learning for Computational Fluid Dynamics: A Survey [51.87875066383221]
  This paper introduces fundamental concepts, traditional methods, and benchmark datasets, then examine the various roles Machine Learning plays in improving CFD. We highlight real-world applications of ML for CFD in critical scientific and engineering disciplines, including aerodynamics, combustion, atmosphere & ocean science, biology fluid, plasma, symbolic regression, and reduced order modeling. We draw the conclusion that ML is poised to significantly transform CFD research by enhancing simulation accuracy, reducing computational time, and enabling more complex analyses of fluid dynamics.
  arXiv  Detail & Related papers  (2024-08-22T07:33:11Z)
• Continual Learning with Pre-Trained Models: A Survey [61.97613090666247]
  Continual Learning aims to overcome the catastrophic forgetting of former knowledge when learning new ones. This paper presents a comprehensive survey of the latest advancements in PTM-based CL.
  arXiv  Detail & Related papers  (2024-01-29T18:27:52Z)
• Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy [0.2796197251957244]
  Catastrophic Forgetting (CF) can lead to a significant loss of accuracy in Deep Learning models. CF was first observed by McCloskey and Cohen in 1989 and remains an active research topic. This article surveys recent studies that tackle CF in modern Deep Learning models that use gradient descent as their learning algorithm.
  arXiv  Detail & Related papers  (2023-12-16T22:24:54Z)
• Constructing Sample-to-Class Graph for Few-Shot Class-Incremental Learning [10.111587226277647]
  Few-shot class-incremental learning (FSCIL) aims to build machine learning model that can continually learn new concepts from a few data samples. In this paper, we propose a Sample-to-Class (S2C) graph learning method for FSCIL.
  arXiv  Detail & Related papers  (2023-10-31T08:38:14Z)
• Few-shot Class-incremental Learning: A Survey [16.729567512584822]
  Few-shot Class-Incremental Learning (FSCIL) presents a unique challenge in Machine Learning (ML) This paper aims to provide a comprehensive and systematic review of FSCIL.
  arXiv  Detail & Related papers  (2023-08-13T13:01:21Z)
• Deep networks for system identification: a Survey [56.34005280792013]
  System identification learns mathematical descriptions of dynamic systems from input-output data. Main aim of the identified model is to predict new data from previous observations. We discuss architectures commonly adopted in the literature, like feedforward, convolutional, and recurrent networks.
  arXiv  Detail & Related papers  (2023-01-30T12:38:31Z)
• Recent Few-Shot Object Detection Algorithms: A Survey with Performance Comparison [54.357707168883024]
  Few-Shot Object Detection (FSOD) mimics the humans' ability of learning to learn. FSOD intelligently transfers the learned generic object knowledge from the common heavy-tailed, to the novel long-tailed object classes. We give an overview of FSOD, including the problem definition, common datasets, and evaluation protocols.
  arXiv  Detail & Related papers  (2022-03-27T04:11:28Z)
• What Makes Good Contrastive Learning on Small-Scale Wearable-based Tasks? [59.51457877578138]
  We study contrastive learning on the wearable-based activity recognition task. This paper presents an open-source PyTorch library textttCL-HAR, which can serve as a practical tool for researchers.
  arXiv  Detail & Related papers  (2022-02-12T06:10:15Z)
• Enhancing Identification of Structure Function of Academic Articles Using Contextual Information [6.28532577139029]
  This paper takes articles of the ACL conference as the corpus to identify the structure function of academic articles. We employ the traditional machine learning models and deep learning models to construct the classifiers based on various feature input. Inspired by (2), this paper introduces contextual information into the deep learning models and achieved significant results.
  arXiv  Detail & Related papers  (2021-11-28T11:21:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.