Novelty-Prepared Few-Shot Classification

• URL: http://arxiv.org/abs/2003.00497v1
• Date: Sun, 1 Mar 2020 14:44:29 GMT
• Title: Novelty-Prepared Few-Shot Classification
• Authors: Chao Wang, Ruo-Ze Liu, Han-Jia Ye, Yang Yu
• Abstract summary: We propose to use a novelty-prepared loss function, called self-compacting softmax loss (SSL), for few-shot classification. In experiments on CUB-200-2011 and mini-ImageNet datasets, we show that SSL leads to significant improvement of the state-of-the-art performance.
• Score: 24.42397780877619
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Few-shot classification algorithms can alleviate the data scarceness issue, which is vital in many real-world problems, by adopting models pre-trained from abundant data in other domains. However, the pre-training process was commonly unaware of the future adaptation to other concept classes. We disclose that a classically fully trained feature extractor can leave little embedding space for unseen classes, which keeps the model from well-fitting the new classes. In this work, we propose to use a novelty-prepared loss function, called self-compacting softmax loss (SSL), for few-shot classification. The SSL can prevent the full occupancy of the embedding space. Thus the model is more prepared to learn new classes. In experiments on CUB-200-2011 and mini-ImageNet datasets, we show that SSL leads to significant improvement of the state-of-the-art performance. This work may shed some light on considering the model capacity for few-shot classification tasks.

Related papers

• Less is More: On the Feature Redundancy of Pretrained Models When Transferring to Few-shot Tasks [120.23328563831704]
  Transferring a pretrained model to a downstream task can be as easy as conducting linear probing with target data. We show that, for linear probing, the pretrained features can be extremely redundant when the downstream data is scarce.
  arXiv  Detail & Related papers  (2023-10-05T19:00:49Z)
• Image-free Classifier Injection for Zero-Shot Classification [72.66409483088995]
  Zero-shot learning models achieve remarkable results on image classification for samples from classes that were not seen during training. We aim to equip pre-trained models with zero-shot classification capabilities without the use of image data. We achieve this with our proposed Image-free Injection with Semantics (ICIS)
  arXiv  Detail & Related papers  (2023-08-21T09:56:48Z)
• RanPAC: Random Projections and Pre-trained Models for Continual Learning [59.07316955610658]
  Continual learning (CL) aims to learn different tasks (such as classification) in a non-stationary data stream without forgetting old ones. We propose a concise and effective approach for CL with pre-trained models.
  arXiv  Detail & Related papers  (2023-07-05T12:49:02Z)
• GMM-IL: Image Classification using Incrementally Learnt, Independent Probabilistic Models for Small Sample Sizes [0.4511923587827301]
  We present a novel two stage architecture which couples visual feature learning with probabilistic models to represent each class. We outperform a benchmark of an equivalent network with a Softmax head, obtaining increased accuracy for sample sizes smaller than 12 and increased weighted F1 score for 3 imbalanced class profiles.
  arXiv  Detail & Related papers  (2022-12-01T15:19:42Z)
• Attaining Class-level Forgetting in Pretrained Model using Few Samples [18.251805180282346]
  In the future, some classes may become restricted due to privacy/ethical concerns. We propose a novel approach to address this problem without affecting the model's prediction power for the remaining classes.
  arXiv  Detail & Related papers  (2022-10-19T15:36:01Z)
• DATA: Domain-Aware and Task-Aware Pre-training [94.62676913928831]
  We present DATA, a simple yet effective NAS approach specialized for self-supervised learning (SSL) Our method achieves promising results across a wide range of computation costs on downstream tasks, including image classification, object detection and semantic segmentation.
  arXiv  Detail & Related papers  (2022-03-17T02:38:49Z)
• Dynamic VAEs with Generative Replay for Continual Zero-shot Learning [1.90365714903665]
  This paper proposes a novel continual zero-shot learning (DVGR-CZSL) model that grows in size with each task and uses generative replay to update itself with previously learned classes to avoid forgetting. We show our method is superior in task sequentially learning with ZSL(Zero-Shot Learning)
  arXiv  Detail & Related papers  (2021-04-26T10:56:43Z)
• Few-Shot Lifelong Learning [35.05196800623617]
  Few-Shot Lifelong Learning enables deep learning models to perform lifelong/continual learning on few-shot data. Our method selects very few parameters from the model for training every new set of classes instead of training the full model. We experimentally show that our method significantly outperforms existing methods on the miniImageNet, CIFAR-100, and CUB-200 datasets.
  arXiv  Detail & Related papers  (2021-03-01T13:26:57Z)
• Adversarial Training of Variational Auto-encoders for Continual Zero-shot Learning [1.90365714903665]
  We present a hybrid network that consists of a shared VAE module to hold information of all tasks and task-specific private VAE modules for each task. The model's size grows with each task to prevent catastrophic forgetting of task-specific skills. We show our method is superior on class sequentially learning with ZSL(Zero-Shot Learning) and GZSL(Generalized Zero-Shot Learning)
  arXiv  Detail & Related papers  (2021-02-07T11:21:24Z)
• Fine-grained Angular Contrastive Learning with Coarse Labels [72.80126601230447]
  We introduce a novel 'Angular normalization' module that allows to effectively combine supervised and self-supervised contrastive pre-training. This work will help to pave the way for future research on this new, challenging, and very practical topic of C2FS classification.
  arXiv  Detail & Related papers  (2020-12-07T08:09:02Z)
• Learning Adaptive Embedding Considering Incremental Class [55.21855842960139]
  Class-Incremental Learning (CIL) aims to train a reliable model with the streaming data, which emerges unknown classes sequentially. Different from traditional closed set learning, CIL has two main challenges: 1) Novel class detection. After the novel classes are detected, the model needs to be updated without re-training using entire previous data.
  arXiv  Detail & Related papers  (2020-08-31T04:11:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.