Brain-inspired analogical mixture prototypes for few-shot class-incremental learning

• URL: http://arxiv.org/abs/2502.18923v1
• Date: Wed, 26 Feb 2025 08:19:55 GMT
• Title: Brain-inspired analogical mixture prototypes for few-shot class-incremental learning
• Authors: Wanyi Li, Wei Wei, Yongkang Luo, Peng Wang,
• Abstract summary: Few-shot class-incremental learning (FSCIL) poses significant challenges for artificial neural networks.<n>We propose a novel approach called Brain-inspired Analogical Mixture Prototypes (BAMP)<n>BAMP has three components: mixed feature learning, statistical analogy, and soft voting.
• Score: 10.78374585352766
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Few-shot class-incremental learning (FSCIL) poses significant challenges for artificial neural networks due to the need to efficiently learn from limited data while retaining knowledge of previously learned tasks. Inspired by the brain's mechanisms for categorization and analogical learning, we propose a novel approach called Brain-inspired Analogical Mixture Prototypes (BAMP). BAMP has three components: mixed prototypical feature learning, statistical analogy, and soft voting. Starting from a pre-trained Vision Transformer (ViT), mixed prototypical feature learning represents each class using a mixture of prototypes and fine-tunes these representations during the base session. The statistical analogy calibrates the mean and covariance matrix of prototypes for new classes according to similarity to the base classes, and computes classification score with Mahalanobis distance. Soft voting combines both merits of statistical analogy and an off-shelf FSCIL method. Our experiments on benchmark datasets demonstrate that BAMP outperforms state-of-the-art on both traditional big start FSCIL setting and challenging small start FSCIL setting. The study suggests that brain-inspired analogical mixture prototypes can alleviate catastrophic forgetting and over-fitting problems in FSCIL.

Related papers

• AM-MTEEG: Multi-task EEG classification based on impulsive associative memory [6.240145569484483]
  We propose a multi-task (MT) classification model, called AM-MTEEG, inspired by the principles of learning and memory in the human hippocampus. The model treats the EEG classification of each individual as an independent task and facilitates feature sharing across individuals. Experimental results in two BCI competition datasets show that our model improves average accuracy compared to state-of-the-art models.
  arXiv  Detail & Related papers  (2024-09-27T01:33:45Z)
• The Languini Kitchen: Enabling Language Modelling Research at Different Scales of Compute [66.84421705029624]
  We introduce an experimental protocol that enables model comparisons based on equivalent compute, measured in accelerator hours. We pre-process an existing large, diverse, and high-quality dataset of books that surpasses existing academic benchmarks in quality, diversity, and document length. This work also provides two baseline models: a feed-forward model derived from the GPT-2 architecture and a recurrent model in the form of a novel LSTM with ten-fold throughput.
  arXiv  Detail & Related papers  (2023-09-20T10:31:17Z)
• Self-Evolution Learning for Mixup: Enhance Data Augmentation on Few-Shot Text Classification Tasks [75.42002070547267]
  We propose a self evolution learning (SE) based mixup approach for data augmentation in text classification. We introduce a novel instance specific label smoothing approach, which linearly interpolates the model's output and one hot labels of the original samples to generate new soft for label mixing up.
  arXiv  Detail & Related papers  (2023-05-22T23:43:23Z)
• Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning [120.53458753007851]
  Few-shot class-incremental learning (FSCIL) has been a challenging problem as only a few training samples are accessible for each novel class in the new sessions. We deal with this misalignment dilemma in FSCIL inspired by the recently discovered phenomenon named neural collapse. We propose a neural collapse inspired framework for FSCIL. Experiments on the miniImageNet, CUB-200, and CIFAR-100 datasets demonstrate that our proposed framework outperforms the state-of-the-art performances.
  arXiv  Detail & Related papers  (2023-02-06T18:39:40Z)
• Learning Primitive-aware Discriminative Representations for Few-shot Learning [28.17404445820028]
  Few-shot learning aims to learn a classifier that can be easily adapted to recognize novel classes with only a few labeled examples. We propose a Primitive Mining and Reasoning Network (PMRN) to learn primitive-aware representations. Our method achieves state-of-the-art results on six standard benchmarks.
  arXiv  Detail & Related papers  (2022-08-20T16:22:22Z)
• Partner-Assisted Learning for Few-Shot Image Classification [54.66864961784989]
  Few-shot Learning has been studied to mimic human visual capabilities and learn effective models without the need of exhaustive human annotation. In this paper, we focus on the design of training strategy to obtain an elemental representation such that the prototype of each novel class can be estimated from a few labeled samples. We propose a two-stage training scheme, which first trains a partner encoder to model pair-wise similarities and extract features serving as soft-anchors, and then trains a main encoder by aligning its outputs with soft-anchors while attempting to maximize classification performance.
  arXiv  Detail & Related papers  (2021-09-15T22:46:19Z)
• A hybrid model-based and learning-based approach for classification using limited number of training samples [13.60714541247498]
  In this paper, a hybrid classification method -- HyPhyLearn -- is proposed that exploits both the physics-based statistical models and the learning-based classifiers. The proposed solution is based on the conjecture that HyPhyLearn would alleviate the challenges associated with the individual approaches of learning-based and statistical model-based classifiers.
  arXiv  Detail & Related papers  (2021-06-25T05:19:50Z)
• No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data [78.69828864672978]
  A central challenge in training classification models in the real-world federated system is learning with non-IID data. We propose a novel and simple algorithm called Virtual Representations (CCVR), which adjusts the classifier using virtual representations sampled from an approximated ssian mixture model. Experimental results demonstrate that CCVR state-of-the-art performance on popular federated learning benchmarks including CIFAR-10, CIFAR-100, and CINIC-10.
  arXiv  Detail & Related papers  (2021-06-09T12:02:29Z)
• A Novel Adaptive Minority Oversampling Technique for Improved Classification in Data Imbalanced Scenarios [23.257891827728827]
  Imbalance in the proportion of training samples belonging to different classes often poses performance degradation of conventional classifiers. We propose a novel three step technique to address imbalanced data.
  arXiv  Detail & Related papers  (2021-03-24T09:58:02Z)
• Few-shot Action Recognition with Prototype-centered Attentive Learning [88.10852114988829]
  Prototype-centered Attentive Learning (PAL) model composed of two novel components. First, a prototype-centered contrastive learning loss is introduced to complement the conventional query-centered learning objective. Second, PAL integrates a attentive hybrid learning mechanism that can minimize the negative impacts of outliers.
  arXiv  Detail & Related papers  (2021-01-20T11:48:12Z)
• An in-depth comparison of methods handling mixed-attribute data for general fuzzy min-max neural network [9.061408029414455]
  We will compare and assess three main methods of handling datasets with mixed features. The experimental results showed that the target and James-Stein are appropriate categorical encoding methods for learning algorithms of GFMM models. The combination of GFMM neural networks and decision trees is a flexible way to enhance the classification performance of GFMM models on datasets with the mixed features.
  arXiv  Detail & Related papers  (2020-09-01T05:12:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.