Tensor feature hallucination for few-shot learning
- URL: http://arxiv.org/abs/2106.05321v1
- Date: Wed, 9 Jun 2021 18:25:08 GMT
- Title: Tensor feature hallucination for few-shot learning
- Authors: Michalis Lazarou, Yannis Avrithis, Tania Stathaki
- Abstract summary: Few-shot classification addresses the challenge of classifying examples given limited supervision and limited data.
Previous works on synthetic data generation for few-shot classification focus on exploiting complex models.
We investigate how a simple and straightforward synthetic data generation method can be used effectively.
- Score: 17.381648488344222
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Few-shot classification addresses the challenge of classifying examples given
not just limited supervision but limited data as well. An attractive solution
is synthetic data generation. However, most such methods are overly
sophisticated, focusing on high-quality, realistic data in the input space. It
is unclear whether adapting them to the few-shot regime and using them for the
downstream task of classification is the right approach. Previous works on
synthetic data generation for few-shot classification focus on exploiting
complex models, e.g. a Wasserstein GAN with multiple regularizers or a network
that transfers latent diversities from known to novel classes.
We follow a different approach and investigate how a simple and
straightforward synthetic data generation method can be used effectively. We
make two contributions, namely we show that: (1) using a simple loss function
is more than enough for training a feature generator in the few-shot setting;
and (2) learning to generate tensor features instead of vector features is
superior. Extensive experiments on miniImagenet, CUB and CIFAR-FS datasets show
that our method sets a new state of the art, outperforming more sophisticated
few-shot data augmentation methods.
Related papers
- Enhancing Consistency and Mitigating Bias: A Data Replay Approach for
Incremental Learning [100.7407460674153]
Deep learning systems are prone to catastrophic forgetting when learning from a sequence of tasks.
To mitigate the problem, a line of methods propose to replay the data of experienced tasks when learning new tasks.
However, it is not expected in practice considering the memory constraint or data privacy issue.
As a replacement, data-free data replay methods are proposed by inverting samples from the classification model.
arXiv Detail & Related papers (2024-01-12T12:51:12Z) - PILoRA: Prototype Guided Incremental LoRA for Federated Class-Incremental Learning [41.984652077669104]
Experimental results on standard datasets indicate that our method outperforms the state-of-the-art approaches significantly.
Our method exhibits strong robustness and superiority in different settings and degrees of data heterogeneity.
arXiv Detail & Related papers (2024-01-04T06:46:19Z) - Improved Distribution Matching for Dataset Condensation [91.55972945798531]
We propose a novel dataset condensation method based on distribution matching.
Our simple yet effective method outperforms most previous optimization-oriented methods with much fewer computational resources.
arXiv Detail & Related papers (2023-07-19T04:07:33Z) - Few-Shot Non-Parametric Learning with Deep Latent Variable Model [50.746273235463754]
We propose Non-Parametric learning by Compression with Latent Variables (NPC-LV)
NPC-LV is a learning framework for any dataset with abundant unlabeled data but very few labeled ones.
We show that NPC-LV outperforms supervised methods on all three datasets on image classification in low data regime.
arXiv Detail & Related papers (2022-06-23T09:35:03Z) - Condensing Graphs via One-Step Gradient Matching [50.07587238142548]
We propose a one-step gradient matching scheme, which performs gradient matching for only one single step without training the network weights.
Our theoretical analysis shows this strategy can generate synthetic graphs that lead to lower classification loss on real graphs.
In particular, we are able to reduce the dataset size by 90% while approximating up to 98% of the original performance.
arXiv Detail & Related papers (2022-06-15T18:20:01Z) - Learning to Generate Synthetic Training Data using Gradient Matching and
Implicit Differentiation [77.34726150561087]
This article explores various data distillation techniques that can reduce the amount of data required to successfully train deep networks.
Inspired by recent ideas, we suggest new data distillation techniques based on generative teaching networks, gradient matching, and the Implicit Function Theorem.
arXiv Detail & Related papers (2022-03-16T11:45:32Z) - No Fear of Heterogeneity: Classifier Calibration for Federated Learning
with Non-IID Data [78.69828864672978]
A central challenge in training classification models in the real-world federated system is learning with non-IID data.
We propose a novel and simple algorithm called Virtual Representations (CCVR), which adjusts the classifier using virtual representations sampled from an approximated ssian mixture model.
Experimental results demonstrate that CCVR state-of-the-art performance on popular federated learning benchmarks including CIFAR-10, CIFAR-100, and CINIC-10.
arXiv Detail & Related papers (2021-06-09T12:02:29Z) - Few-shot learning via tensor hallucination [17.381648488344222]
Few-shot classification addresses the challenge of classifying examples given only limited labeled data.
We show that using a simple loss function is more than enough for training a feature generator in the few-shot setting.
Our method sets a new state of the art, outperforming more sophisticated few-shot data augmentation methods.
arXiv Detail & Related papers (2021-04-19T17:30:33Z) - Few-Shot Incremental Learning with Continually Evolved Classifiers [46.278573301326276]
Few-shot class-incremental learning (FSCIL) aims to design machine learning algorithms that can continually learn new concepts from a few data points.
The difficulty lies in that limited data from new classes not only lead to significant overfitting issues but also exacerbate the notorious catastrophic forgetting problems.
We propose a Continually Evolved CIF ( CEC) that employs a graph model to propagate context information between classifiers for adaptation.
arXiv Detail & Related papers (2021-04-07T10:54:51Z) - Multimodal Prototypical Networks for Few-shot Learning [20.100480009813953]
Cross-modal feature generation framework is used to enrich the low populated embedding space in few-shot scenarios.
We show that in such cases nearest neighbor classification is a viable approach and outperform state-of-the-art single-modal and multimodal few-shot learning methods.
arXiv Detail & Related papers (2020-11-17T19:32:59Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.