POODLE: Improving Few-shot Learning via Penalizing Out-of-Distribution Samples

• URL: http://arxiv.org/abs/2206.04679v1
• Date: Wed, 8 Jun 2022 18:59:21 GMT
• Title: POODLE: Improving Few-shot Learning via Penalizing Out-of-Distribution Samples
• Authors: Duong H. Le, Khoi D. Nguyen, Khoi Nguyen, Quoc-Huy Tran, Rang Nguyen, Binh-Son Hua
• Abstract summary: We propose to use out-of-distribution samples, i.e., unlabeled samples coming from outside the target classes, to improve few-shot learning. Our approach is simple to implement, agnostic to feature extractors, lightweight without any additional cost for pre-training, and applicable to both inductive and transductive settings.
• Score: 19.311470287767385
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In this work, we propose to use out-of-distribution samples, i.e., unlabeled samples coming from outside the target classes, to improve few-shot learning. Specifically, we exploit the easily available out-of-distribution samples to drive the classifier to avoid irrelevant features by maximizing the distance from prototypes to out-of-distribution samples while minimizing that of in-distribution samples (i.e., support, query data). Our approach is simple to implement, agnostic to feature extractors, lightweight without any additional cost for pre-training, and applicable to both inductive and transductive settings. Extensive experiments on various standard benchmarks demonstrate that the proposed method consistently improves the performance of pretrained networks with different architectures.

Related papers

• Self-Guided Generation of Minority Samples Using Diffusion Models [57.319845580050924]
  We present a novel approach for generating minority samples that live on low-density regions of a data manifold. Our framework is built upon diffusion models, leveraging the principle of guided sampling. Experiments on benchmark real datasets demonstrate that our approach can greatly improve the capability of creating realistic low-likelihood minority instances.
  arXiv  Detail & Related papers  (2024-07-16T10:03:29Z)
• How Low Can You Go? Surfacing Prototypical In-Distribution Samples for Unsupervised Anomaly Detection [48.30283806131551]
  We show that UAD with extremely few training samples can already match -- and in some cases even surpass -- the performance of training with the whole training dataset. We propose an unsupervised method to reliably identify prototypical samples to further boost UAD performance.
  arXiv  Detail & Related papers  (2023-12-06T15:30:47Z)
• Data Pruning via Moving-one-Sample-out [61.45441981346064]
  We propose a novel data-pruning approach called moving-one-sample-out (MoSo) MoSo aims to identify and remove the least informative samples from the training set. Experimental results demonstrate that MoSo effectively mitigates severe performance degradation at high pruning ratios.
  arXiv  Detail & Related papers  (2023-10-23T08:00:03Z)
• Multisample Flow Matching: Straightening Flows with Minibatch Couplings [38.82598694134521]
  Simulation-free methods for training continuous-time generative models construct probability paths that go between noise distributions and individual data samples. We propose Multisample Flow Matching, a more general framework that uses non-trivial couplings between data and noise samples. We show that our proposed methods improve sample consistency on downsampled ImageNet data sets, and lead to better low-cost sample generation.
  arXiv  Detail & Related papers  (2023-04-28T11:33:08Z)
• Squeezing Backbone Feature Distributions to the Max for Efficient Few-Shot Learning [3.1153758106426603]
  Few-shot classification is a challenging problem due to the uncertainty caused by using few labelled samples. We propose a novel transfer-based method which aims at processing the feature vectors so that they become closer to Gaussian-like distributions. In the case of transductive few-shot learning where unlabelled test samples are available during training, we also introduce an optimal-transport inspired algorithm to boost even further the achieved performance.
  arXiv  Detail & Related papers  (2021-10-18T16:29:17Z)
• Lightweight Detection of Out-of-Distribution and Adversarial Samples via Channel Mean Discrepancy [14.103271496247551]
  We introduce Channel Mean Discrepancy (CMD), a model-agnostic distance metric for evaluating the statistics of features extracted by classification models. We experimentally demonstrate that CMD magnitude is significantly smaller for legitimate samples than for OOD and adversarial samples. Preliminary results show that our simple yet effective method outperforms several state-of-the-art approaches to detecting OOD and adversarial samples.
  arXiv  Detail & Related papers  (2021-04-23T04:15:53Z)
• Jo-SRC: A Contrastive Approach for Combating Noisy Labels [58.867237220886885]
  We propose a noise-robust approach named Jo-SRC (Joint Sample Selection and Model Regularization based on Consistency) Specifically, we train the network in a contrastive learning manner. Predictions from two different views of each sample are used to estimate its "likelihood" of being clean or out-of-distribution.
  arXiv  Detail & Related papers  (2021-03-24T07:26:07Z)
• Multi-Scale Positive Sample Refinement for Few-Shot Object Detection [61.60255654558682]
  Few-shot object detection (FSOD) helps detectors adapt to unseen classes with few training instances. We propose a Multi-scale Positive Sample Refinement (MPSR) approach to enrich object scales in FSOD. MPSR generates multi-scale positive samples as object pyramids and refines the prediction at various scales.
  arXiv  Detail & Related papers  (2020-07-18T09:48:29Z)
• CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances [77.28192419848901]
  We propose a simple, yet effective method named contrasting shifted instances (CSI) In addition to contrasting a given sample with other instances as in conventional contrastive learning methods, our training scheme contrasts the sample with distributionally-shifted augmentations of itself. Our experiments demonstrate the superiority of our method under various novelty detection scenarios.
  arXiv  Detail & Related papers  (2020-07-16T08:32:56Z)
• Robust Federated Learning: The Case of Affine Distribution Shifts [41.27887358989414]
  We develop a robust federated learning algorithm that achieves satisfactory performance against distribution shifts in users' samples. We show that an affine distribution shift indeed suffices to significantly decrease the performance of the learnt classifier in a new test user.
  arXiv  Detail & Related papers  (2020-06-16T03:43:59Z)
• Learning Diverse Representations for Fast Adaptation to Distribution Shift [78.83747601814669]
  We present a method for learning multiple models, incorporating an objective that pressures each to learn a distinct way to solve the task. We demonstrate our framework's ability to facilitate rapid adaptation to distribution shift.
  arXiv  Detail & Related papers  (2020-06-12T12:23:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.