Adaptive Distribution Calibration for Few-Shot Learning with Hierarchical Optimal Transport

• URL: http://arxiv.org/abs/2210.04144v1
• Date: Sun, 9 Oct 2022 02:32:57 GMT
• Title: Adaptive Distribution Calibration for Few-Shot Learning with Hierarchical Optimal Transport
• Authors: Dandan Guo, Long Tian, He Zhao, Mingyuan Zhou, Hongyuan Zha
• Abstract summary: We propose a novel distribution calibration method by learning the adaptive weight matrix between novel samples and base classes. Experimental results on standard benchmarks demonstrate that our proposed plug-and-play model outperforms competing approaches.
• Score: 78.9167477093745
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Few-shot classification aims to learn a classifier to recognize unseen classes during training, where the learned model can easily become over-fitted based on the biased distribution formed by only a few training examples. A recent solution to this problem is calibrating the distribution of these few sample classes by transferring statistics from the base classes with sufficient examples, where how to decide the transfer weights from base classes to novel classes is the key. However, principled approaches for learning the transfer weights have not been carefully studied. To this end, we propose a novel distribution calibration method by learning the adaptive weight matrix between novel samples and base classes, which is built upon a hierarchical Optimal Transport (H-OT) framework. By minimizing the high-level OT distance between novel samples and base classes, we can view the learned transport plan as the adaptive weight information for transferring the statistics of base classes. The learning of the cost function between a base class and novel class in the high-level OT leads to the introduction of the low-level OT, which considers the weights of all the data samples in the base class. Experimental results on standard benchmarks demonstrate that our proposed plug-and-play model outperforms competing approaches and owns desired cross-domain generalization ability, indicating the effectiveness of the learned adaptive weights.

Related papers

• Covariance-based Space Regularization for Few-shot Class Incremental Learning [25.435192867105552]
  Few-shot Class Incremental Learning (FSCIL) requires the model to continually learn new classes with limited labeled data. Due to the limited data in incremental sessions, models are prone to overfitting new classes and suffering catastrophic forgetting of base classes. Recent advancements resort to prototype-based approaches to constrain the base class distribution and learn discriminative representations of new classes.
  arXiv  Detail & Related papers  (2024-11-02T08:03:04Z)
• Adaptive Margin Global Classifier for Exemplar-Free Class-Incremental Learning [3.4069627091757178]
  Existing methods mainly focus on handling biased learning. We introduce a Distribution-Based Global (DBGC) to avoid bias factors in existing methods, such as data imbalance and sampling. More importantly, the compromised distributions of old classes are simulated via a simple operation, variance (VE). This loss is proven equivalent to an Adaptive Margin Softmax Cross Entropy (AMarX)
  arXiv  Detail & Related papers  (2024-09-20T07:07:23Z)
• Twice Class Bias Correction for Imbalanced Semi-Supervised Learning [59.90429949214134]
  We introduce a novel approach called textbfTwice textbfClass textbfBias textbfCorrection (textbfTCBC) We estimate the class bias of the model parameters during the training process. We apply a secondary correction to the model's pseudo-labels for unlabeled samples.
  arXiv  Detail & Related papers  (2023-12-27T15:06:36Z)
• Learnable Distribution Calibration for Few-Shot Class-Incremental Learning [122.2241120474278]
  Few-shot class-incremental learning (FSCIL) faces challenges of memorizing old class distributions and estimating new class distributions given few training samples. We propose a learnable distribution calibration (LDC) approach, with the aim to systematically solve these two challenges using a unified framework.
  arXiv  Detail & Related papers  (2022-10-01T09:40:26Z)
• Learning to Re-weight Examples with Optimal Transport for Imbalanced Classification [74.62203971625173]
  Imbalanced data pose challenges for deep learning based classification models. One of the most widely-used approaches for tackling imbalanced data is re-weighting. We propose a novel re-weighting method based on optimal transport (OT) from a distributional point of view.
  arXiv  Detail & Related papers  (2022-08-05T01:23:54Z)
• Class-Incremental Learning with Strong Pre-trained Models [97.84755144148535]
  Class-incremental learning (CIL) has been widely studied under the setting of starting from a small number of classes (base classes) We explore an understudied real-world setting of CIL that starts with a strong model pre-trained on a large number of base classes. Our proposed method is robust and generalizes to all analyzed CIL settings.
  arXiv  Detail & Related papers  (2022-04-07T17:58:07Z)
• CMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep Learning [55.733193075728096]
  Modern deep neural networks can easily overfit to biased training data containing corrupted labels or class imbalance. Sample re-weighting methods are popularly used to alleviate this data bias issue. We propose a meta-model capable of adaptively learning an explicit weighting scheme directly from data.
  arXiv  Detail & Related papers  (2022-02-11T13:49:51Z)
• Variational Feature Disentangling for Fine-Grained Few-Shot Classification [30.350307891161865]
  Fine-grained few-shot recognition often suffers from the problem of training data scarcity for novel categories. In this paper, we focus one enlarging the intra-class variance of the unseen class to improve few-shot classification performance.
  arXiv  Detail & Related papers  (2020-10-07T08:13:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.