Automatic Combination of Sample Selection Strategies for Few-Shot Learning

• URL: http://arxiv.org/abs/2402.03038v1
• Date: Mon, 5 Feb 2024 14:23:43 GMT
• Title: Automatic Combination of Sample Selection Strategies for Few-Shot Learning
• Authors: Branislav Pecher, Ivan Srba, Maria Bielikova, Joaquin Vanschoren
• Abstract summary: In few-shot learning, the limited number of samples used to train a model have a significant impact on the overall success. We investigate the impact of 20 sample selection strategies on the performance of 5 few-shot learning approaches over 8 image and 6 text datasets.
• Score: 8.741702582225987
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In few-shot learning, such as meta-learning, few-shot fine-tuning or in-context learning, the limited number of samples used to train a model have a significant impact on the overall success. Although a large number of sample selection strategies exist, their impact on the performance of few-shot learning is not extensively known, as most of them have been so far evaluated in typical supervised settings only. In this paper, we thoroughly investigate the impact of 20 sample selection strategies on the performance of 5 few-shot learning approaches over 8 image and 6 text datasets. In addition, we propose a new method for automatic combination of sample selection strategies (ACSESS) that leverages the strengths and complementary information of the individual strategies. The experimental results show that our method consistently outperforms the individual selection strategies, as well as the recently proposed method for selecting support examples for in-context learning. We also show a strong modality, dataset and approach dependence for the majority of strategies as well as their dependence on the number of shots - demonstrating that the sample selection strategies play a significant role for lower number of shots, but regresses to random selection at higher number of shots.

Related papers

• Use Random Selection for Now: Investigation of Few-Shot Selection Strategies in LLM-based Text Augmentation for Classification [6.273933281069326]
  generative large language models (LLMs) are increasingly used for data augmentation tasks. We compare sample selection strategies existing in few-shot learning literature and investigate their effects in LLM-based textual augmentation. Results indicate, that while some informed'' selection strategies increase the performance of models, it happens only seldom and with marginal performance increases.
  arXiv  Detail & Related papers  (2024-10-14T17:30:08Z)
• Model-Free Active Exploration in Reinforcement Learning [53.786439742572995]
  We study the problem of exploration in Reinforcement Learning and present a novel model-free solution. Our strategy is able to identify efficient policies faster than state-of-the-art exploration approaches.
  arXiv  Detail & Related papers  (2024-06-30T19:00:49Z)
• Diversified Batch Selection for Training Acceleration [68.67164304377732]
  A prevalent research line, known as online batch selection, explores selecting informative subsets during the training process. vanilla reference-model-free methods involve independently scoring and selecting data in a sample-wise manner. We propose Diversified Batch Selection (DivBS), which is reference-model-free and can efficiently select diverse and representative samples.
  arXiv  Detail & Related papers  (2024-06-07T12:12:20Z)
• What Makes Good Few-shot Examples for Vision-Language Models? [29.620987070958318]
  We introduce two innovative selection methods - Representativeness (REPRE) and Gaussian Monte Carlo (Montecarlo) Our findings demonstrate that both REPRE and Montecarlo significantly surpass both random selection and AL-based strategies in few-shot training scenarios. The research also underscores that these instance selection methods are model-agnostic, offering a versatile enhancement to a wide array of few-shot training methodologies.
  arXiv  Detail & Related papers  (2024-05-22T11:03:33Z)
• ALE: A Simulation-Based Active Learning Evaluation Framework for the Parameter-Driven Comparison of Query Strategies for NLP [3.024761040393842]
  Active Learning (AL) proposes promising data points to annotators they annotate next instead of a subsequent or random sample. This method is supposed to save annotation effort while maintaining model performance. We introduce a reproducible active learning evaluation framework for the comparative evaluation of AL strategies in NLP.
  arXiv  Detail & Related papers  (2023-08-01T10:42:11Z)
• Budget-aware Few-shot Learning via Graph Convolutional Network [56.41899553037247]
  This paper tackles the problem of few-shot learning, which aims to learn new visual concepts from a few examples. A common problem setting in few-shot classification assumes random sampling strategy in acquiring data labels. We introduce a new budget-aware few-shot learning problem that aims to learn novel object categories.
  arXiv  Detail & Related papers  (2022-01-07T02:46:35Z)
• An Investigation of Replay-based Approaches for Continual Learning [79.0660895390689]
  Continual learning (CL) is a major challenge of machine learning (ML) and describes the ability to learn several tasks sequentially without catastrophic forgetting (CF) Several solution classes have been proposed, of which so-called replay-based approaches seem very promising due to their simplicity and robustness. We empirically investigate replay-based approaches of continual learning and assess their potential for applications.
  arXiv  Detail & Related papers  (2021-08-15T15:05:02Z)
• Jo-SRC: A Contrastive Approach for Combating Noisy Labels [58.867237220886885]
  We propose a noise-robust approach named Jo-SRC (Joint Sample Selection and Model Regularization based on Consistency) Specifically, we train the network in a contrastive learning manner. Predictions from two different views of each sample are used to estimate its "likelihood" of being clean or out-of-distribution.
  arXiv  Detail & Related papers  (2021-03-24T07:26:07Z)
• Improved active output selection strategy for noisy environments [0.0]
  The test bench time needed for model-based calibration can be reduced with active learning methods for test design. This paper presents an improved strategy for active output selection.
  arXiv  Detail & Related papers  (2021-01-10T08:27:30Z)
• Few-shot Classification via Adaptive Attention [93.06105498633492]
  We propose a novel few-shot learning method via optimizing and fast adapting the query sample representation based on very few reference samples. As demonstrated experimentally, the proposed model achieves state-of-the-art classification results on various benchmark few-shot classification and fine-grained recognition datasets.
  arXiv  Detail & Related papers  (2020-08-06T05:52:59Z)
• SEERL: Sample Efficient Ensemble Reinforcement Learning [20.983016439055188]
  We present a novel training and model selection framework for model-free reinforcement algorithms. We show that learning and selecting an adequately diverse set of policies is required for a good ensemble. Our framework is substantially sample efficient, computationally inexpensive and is seen to outperform state-of-the-art (SOTA) scores in Atari 2600 and Mujoco.
  arXiv  Detail & Related papers  (2020-01-15T10:12:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.