Few-shot Learning via Dependency Maximization and Instance Discriminant
Analysis
- URL: http://arxiv.org/abs/2109.02820v1
- Date: Tue, 7 Sep 2021 02:19:01 GMT
- Title: Few-shot Learning via Dependency Maximization and Instance Discriminant
Analysis
- Authors: Zejiang Hou, Sun-Yuan Kung
- Abstract summary: We study the few-shot learning problem, where a model learns to recognize new objects with extremely few labeled data per category.
We propose a simple approach to exploit unlabeled data accompanying the few-shot task for improving few-shot performance.
- Score: 21.8311401851523
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: We study the few-shot learning (FSL) problem, where a model learns to
recognize new objects with extremely few labeled training data per category.
Most of previous FSL approaches resort to the meta-learning paradigm, where the
model accumulates inductive bias through learning many training tasks so as to
solve a new unseen few-shot task. In contrast, we propose a simple approach to
exploit unlabeled data accompanying the few-shot task for improving few-shot
performance. Firstly, we propose a Dependency Maximization method based on the
Hilbert-Schmidt norm of the cross-covariance operator, which maximizes the
statistical dependency between the embedded feature of those unlabeled data and
their label predictions, together with the supervised loss over the support
set. We then use the obtained model to infer the pseudo-labels for those
unlabeled data. Furthermore, we propose anInstance Discriminant Analysis to
evaluate the credibility of each pseudo-labeled example and select the most
faithful ones into an augmented support set to retrain the model as in the
first step. We iterate the above process until the pseudo-labels for the
unlabeled data becomes stable. Following the standard transductive and
semi-supervised FSL setting, our experiments show that the proposed method
out-performs previous state-of-the-art methods on four widely used benchmarks,
including mini-ImageNet, tiered-ImageNet, CUB, and CIFARFS.
Related papers
- LPLgrad: Optimizing Active Learning Through Gradient Norm Sample Selection and Auxiliary Model Training [2.762397703396293]
Loss Prediction Loss with Gradient Norm (LPLgrad) is designed to quantify model uncertainty effectively and improve the accuracy of image classification tasks.
LPLgrad operates in two distinct phases: (i) em Training Phase aims to predict the loss for input features by jointly training a main model and an auxiliary model.
This dual-model approach enhances the ability to extract complex input features and learn intrinsic patterns from the data effectively.
arXiv Detail & Related papers (2024-11-20T18:12:59Z) - EOL: Transductive Few-Shot Open-Set Recognition by Enhancing Outlier Logits [16.081748213657825]
In Few-Shot Learning, models are trained to recognise unseen objects from a query set, given a few labelled examples from a support set.
In this work, we explore the more nuanced and practical challenge of Open-Set Few-Shot Recognition.
arXiv Detail & Related papers (2024-08-04T15:00:22Z) - Rethinking Classifier Re-Training in Long-Tailed Recognition: A Simple
Logits Retargeting Approach [102.0769560460338]
We develop a simple logits approach (LORT) without the requirement of prior knowledge of the number of samples per class.
Our method achieves state-of-the-art performance on various imbalanced datasets, including CIFAR100-LT, ImageNet-LT, and iNaturalist 2018.
arXiv Detail & Related papers (2024-03-01T03:27:08Z) - MaxMatch: Semi-Supervised Learning with Worst-Case Consistency [149.03760479533855]
We propose a worst-case consistency regularization technique for semi-supervised learning (SSL)
We present a generalization bound for SSL consisting of the empirical loss terms observed on labeled and unlabeled training data separately.
Motivated by this bound, we derive an SSL objective that minimizes the largest inconsistency between an original unlabeled sample and its multiple augmented variants.
arXiv Detail & Related papers (2022-09-26T12:04:49Z) - A Lagrangian Duality Approach to Active Learning [119.36233726867992]
We consider the batch active learning problem, where only a subset of the training data is labeled.
We formulate the learning problem using constrained optimization, where each constraint bounds the performance of the model on labeled samples.
We show, via numerical experiments, that our proposed approach performs similarly to or better than state-of-the-art active learning methods.
arXiv Detail & Related papers (2022-02-08T19:18:49Z) - Dash: Semi-Supervised Learning with Dynamic Thresholding [72.74339790209531]
We propose a semi-supervised learning (SSL) approach that uses unlabeled examples to train models.
Our proposed approach, Dash, enjoys its adaptivity in terms of unlabeled data selection.
arXiv Detail & Related papers (2021-09-01T23:52:29Z) - How to trust unlabeled data? Instance Credibility Inference for Few-Shot
Learning [47.21354101796544]
This paper presents a statistical approach, dubbed Instance Credibility Inference (ICI) to exploit the support of unlabeled instances for few-shot visual recognition.
We rank the credibility of pseudo-labeled instances along the regularization path of their corresponding incidental parameters, and the most trustworthy pseudo-labeled examples are preserved as the augmented labeled instances.
arXiv Detail & Related papers (2020-07-15T03:38:09Z) - Semi-Supervised Learning with Meta-Gradient [123.26748223837802]
We propose a simple yet effective meta-learning algorithm in semi-supervised learning.
We find that the proposed algorithm performs favorably against state-of-the-art methods.
arXiv Detail & Related papers (2020-07-08T08:48:56Z) - Instance Credibility Inference for Few-Shot Learning [45.577880041135785]
Few-shot learning aims to recognize new objects with extremely limited training data for each category.
This paper presents a simple statistical approach, dubbed Instance Credibility Inference (ICI) to exploit the distribution support of unlabeled instances for few-shot learning.
Our simple approach can establish new state-of-the-arts on four widely used few-shot learning benchmark datasets.
arXiv Detail & Related papers (2020-03-26T12:01:15Z) - Meta-Learned Confidence for Few-shot Learning [60.6086305523402]
A popular transductive inference technique for few-shot metric-based approaches, is to update the prototype of each class with the mean of the most confident query examples.
We propose to meta-learn the confidence for each query sample, to assign optimal weights to unlabeled queries.
We validate our few-shot learning model with meta-learned confidence on four benchmark datasets.
arXiv Detail & Related papers (2020-02-27T10:22:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.