A Survey on Few-Shot Class-Incremental Learning
- URL: http://arxiv.org/abs/2304.08130v2
- Date: Tue, 24 Oct 2023 03:08:47 GMT
- Title: A Survey on Few-Shot Class-Incremental Learning
- Authors: Songsong Tian, Lusi Li, Weijun Li, Hang Ran, Xin Ning, Prayag Tiwari
- Abstract summary: Few-shot class-incremental learning (FSCIL) poses a significant challenge for deep neural networks to learn new tasks.
This paper provides a comprehensive survey on FSCIL.
FSCIL has achieved impressive achievements in various fields of computer vision.
- Score: 11.68962265057818
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Large deep learning models are impressive, but they struggle when real-time
data is not available. Few-shot class-incremental learning (FSCIL) poses a
significant challenge for deep neural networks to learn new tasks from just a
few labeled samples without forgetting the previously learned ones. This setup
easily leads to catastrophic forgetting and overfitting problems, severely
affecting model performance. Studying FSCIL helps overcome deep learning model
limitations on data volume and acquisition time, while improving practicality
and adaptability of machine learning models. This paper provides a
comprehensive survey on FSCIL. Unlike previous surveys, we aim to synthesize
few-shot learning and incremental learning, focusing on introducing FSCIL from
two perspectives, while reviewing over 30 theoretical research studies and more
than 20 applied research studies. From the theoretical perspective, we provide
a novel categorization approach that divides the field into five subcategories,
including traditional machine learning methods, meta-learning based methods,
feature and feature space-based methods, replay-based methods, and dynamic
network structure-based methods. We also evaluate the performance of recent
theoretical research on benchmark datasets of FSCIL. From the application
perspective, FSCIL has achieved impressive achievements in various fields of
computer vision such as image classification, object detection, and image
segmentation, as well as in natural language processing and graph. We summarize
the important applications. Finally, we point out potential future research
directions, including applications, problem setups, and theory development.
Overall, this paper offers a comprehensive analysis of the latest advances in
FSCIL from a methodological, performance, and application perspective.
Related papers
- Few Shot Class Incremental Learning using Vision-Language models [24.930246674021525]
In this study, we introduce an innovative few-shot class incremental learning (FSCIL) framework that utilizes language regularizer and subspace regularizer.
Our proposed framework not only empowers the model to embrace novel classes with limited data, but also ensures the preservation of performance on base classes.
arXiv Detail & Related papers (2024-05-02T06:52:49Z) - Continual Learning with Pre-Trained Models: A Survey [61.97613090666247]
Continual Learning aims to overcome the catastrophic forgetting of former knowledge when learning new ones.
This paper presents a comprehensive survey of the latest advancements in PTM-based CL.
arXiv Detail & Related papers (2024-01-29T18:27:52Z) - Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy [0.2796197251957244]
Catastrophic Forgetting (CF) can lead to a significant loss of accuracy in Deep Learning models.
CF was first observed by McCloskey and Cohen in 1989 and remains an active research topic.
This article surveys recent studies that tackle CF in modern Deep Learning models that use gradient descent as their learning algorithm.
arXiv Detail & Related papers (2023-12-16T22:24:54Z) - Constructing Sample-to-Class Graph for Few-Shot Class-Incremental
Learning [10.111587226277647]
Few-shot class-incremental learning (FSCIL) aims to build machine learning model that can continually learn new concepts from a few data samples.
In this paper, we propose a Sample-to-Class (S2C) graph learning method for FSCIL.
arXiv Detail & Related papers (2023-10-31T08:38:14Z) - Few-shot Class-incremental Learning: A Survey [16.729567512584822]
Few-shot Class-Incremental Learning (FSCIL) presents a unique challenge in Machine Learning (ML)
This paper aims to provide a comprehensive and systematic review of FSCIL.
arXiv Detail & Related papers (2023-08-13T13:01:21Z) - Deep networks for system identification: a Survey [56.34005280792013]
System identification learns mathematical descriptions of dynamic systems from input-output data.
Main aim of the identified model is to predict new data from previous observations.
We discuss architectures commonly adopted in the literature, like feedforward, convolutional, and recurrent networks.
arXiv Detail & Related papers (2023-01-30T12:38:31Z) - Recent Few-Shot Object Detection Algorithms: A Survey with Performance
Comparison [54.357707168883024]
Few-Shot Object Detection (FSOD) mimics the humans' ability of learning to learn.
FSOD intelligently transfers the learned generic object knowledge from the common heavy-tailed, to the novel long-tailed object classes.
We give an overview of FSOD, including the problem definition, common datasets, and evaluation protocols.
arXiv Detail & Related papers (2022-03-27T04:11:28Z) - What Makes Good Contrastive Learning on Small-Scale Wearable-based
Tasks? [59.51457877578138]
We study contrastive learning on the wearable-based activity recognition task.
This paper presents an open-source PyTorch library textttCL-HAR, which can serve as a practical tool for researchers.
arXiv Detail & Related papers (2022-02-12T06:10:15Z) - Enhancing Identification of Structure Function of Academic Articles
Using Contextual Information [6.28532577139029]
This paper takes articles of the ACL conference as the corpus to identify the structure function of academic articles.
We employ the traditional machine learning models and deep learning models to construct the classifiers based on various feature input.
Inspired by (2), this paper introduces contextual information into the deep learning models and achieved significant results.
arXiv Detail & Related papers (2021-11-28T11:21:21Z) - A Survey on Concept Factorization: From Shallow to Deep Representation
Learning [104.78577405792592]
Concept Factorization (CF) has attracted a great deal of interests in the areas of machine learning and data mining.
We first re-view the root CF method, and then explore the advancement of CF-based representation learning.
We also introduce the potential application areas of CF-based methods.
arXiv Detail & Related papers (2020-07-31T04:19:14Z) - Few-Shot Class-Incremental Learning [68.75462849428196]
We focus on a challenging but practical few-shot class-incremental learning (FSCIL) problem.
FSCIL requires CNN models to incrementally learn new classes from very few labelled samples, without forgetting the previously learned ones.
We represent the knowledge using a neural gas (NG) network, which can learn and preserve the topology of the feature manifold formed by different classes.
arXiv Detail & Related papers (2020-04-23T03:38:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.