Free Lunch for Few-shot Learning: Distribution Calibration
- URL: http://arxiv.org/abs/2101.06395v2
- Date: Mon, 15 Mar 2021 08:34:18 GMT
- Title: Free Lunch for Few-shot Learning: Distribution Calibration
- Authors: Shuo Yang, Lu Liu, Min Xu
- Abstract summary: We show that a simple logistic regression classifier trained using the features sampled from our calibrated distribution can outperform the state-of-the-art accuracy on two datasets.
- Score: 10.474018806591397
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Learning from a limited number of samples is challenging since the learned
model can easily become overfitted based on the biased distribution formed by
only a few training examples. In this paper, we calibrate the distribution of
these few-sample classes by transferring statistics from the classes with
sufficient examples, then an adequate number of examples can be sampled from
the calibrated distribution to expand the inputs to the classifier. We assume
every dimension in the feature representation follows a Gaussian distribution
so that the mean and the variance of the distribution can borrow from that of
similar classes whose statistics are better estimated with an adequate number
of samples. Our method can be built on top of off-the-shelf pretrained feature
extractors and classification models without extra parameters. We show that a
simple logistic regression classifier trained using the features sampled from
our calibrated distribution can outperform the state-of-the-art accuracy on two
datasets (~5% improvement on miniImageNet compared to the next best). The
visualization of these generated features demonstrates that our calibrated
distribution is an accurate estimation.
Related papers
- Theory on Score-Mismatched Diffusion Models and Zero-Shot Conditional Samplers [49.97755400231656]
We present the first performance guarantee with explicit dimensional general score-mismatched diffusion samplers.
We show that score mismatches result in an distributional bias between the target and sampling distributions, proportional to the accumulated mismatch between the target and training distributions.
This result can be directly applied to zero-shot conditional samplers for any conditional model, irrespective of measurement noise.
arXiv Detail & Related papers (2024-10-17T16:42:12Z) - Probabilistic Contrastive Learning for Long-Tailed Visual Recognition [78.70453964041718]
Longtailed distributions frequently emerge in real-world data, where a large number of minority categories contain a limited number of samples.
Recent investigations have revealed that supervised contrastive learning exhibits promising potential in alleviating the data imbalance.
We propose a novel probabilistic contrastive (ProCo) learning algorithm that estimates the data distribution of the samples from each class in the feature space.
arXiv Detail & Related papers (2024-03-11T13:44:49Z) - Logarithm-transform aided Gaussian Sampling for Few-Shot Learning [0.0]
I propose a novel Gaussian transform, that outperforms existing methods on transforming experimental data into Gaussian-like distributions.
I then utilise this novel transformation for few-shot image classification and show significant gains in performance, while sampling lesser data.
arXiv Detail & Related papers (2023-09-28T10:50:32Z) - Boosting Few-Shot Text Classification via Distribution Estimation [38.99459686893034]
We propose two simple yet effective strategies to estimate the distributions of the novel classes by utilizing unlabeled query samples.
Specifically, we first assume a class or sample follows the Gaussian distribution, and use the original support set and the nearest few query samples.
Then, we augment the labeled samples by sampling from the estimated distribution, which can provide sufficient supervision for training the classification model.
arXiv Detail & Related papers (2023-03-26T05:58:39Z) - Learnable Distribution Calibration for Few-Shot Class-Incremental
Learning [122.2241120474278]
Few-shot class-incremental learning (FSCIL) faces challenges of memorizing old class distributions and estimating new class distributions given few training samples.
We propose a learnable distribution calibration (LDC) approach, with the aim to systematically solve these two challenges using a unified framework.
arXiv Detail & Related papers (2022-10-01T09:40:26Z) - Feature Generation for Long-tail Classification [36.186909933006675]
We show how to generate meaningful features by estimating the tail category's distribution.
We also present a qualitative analysis of generated features using t-SNE visualizations and analyze the nearest neighbors used to calibrate the tail class distributions.
arXiv Detail & Related papers (2021-11-10T21:34:29Z) - Sampling from Arbitrary Functions via PSD Models [55.41644538483948]
We take a two-step approach by first modeling the probability distribution and then sampling from that model.
We show that these models can approximate a large class of densities concisely using few evaluations, and present a simple algorithm to effectively sample from these models.
arXiv Detail & Related papers (2021-10-20T12:25:22Z) - Unrolling Particles: Unsupervised Learning of Sampling Distributions [102.72972137287728]
Particle filtering is used to compute good nonlinear estimates of complex systems.
We show in simulations that the resulting particle filter yields good estimates in a wide range of scenarios.
arXiv Detail & Related papers (2021-10-06T16:58:34Z) - Distributional Reinforcement Learning via Moment Matching [54.16108052278444]
We formulate a method that learns a finite set of statistics from each return distribution via neural networks.
Our method can be interpreted as implicitly matching all orders of moments between a return distribution and its Bellman target.
Experiments on the suite of Atari games show that our method outperforms the standard distributional RL baselines.
arXiv Detail & Related papers (2020-07-24T05:18:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.