Sample Noise Impact on Active Learning

• URL: http://arxiv.org/abs/2109.01372v1
• Date: Fri, 3 Sep 2021 08:36:13 GMT
• Title: Sample Noise Impact on Active Learning
• Authors: Alexandre Abraham and L\'eo Dreyfus-Schmidt
• Abstract summary: This work explores the effect of noisy sample selection in active learning strategies. We show on both synthetic problems and real-life use-cases that knowledge of the sample noise can significantly improve the performance of active learning strategies.
• Score: 77.99796068970569
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This work explores the effect of noisy sample selection in active learning strategies. We show on both synthetic problems and real-life use-cases that knowledge of the sample noise can significantly improve the performance of active learning strategies. Building on prior work, we propose a robust sampler, Incremental Weighted K-Means that brings significant improvement on the synthetic tasks but only a marginal uplift on real-life ones. We hope that the questions raised in this paper are of interest to the community and could open new paths for active learning research.

Related papers

• Hide and Seek in Noise Labels: Noise-Robust Collaborative Active Learning with LLM-Powered Assistance [17.359530437698723]
  Learning from noisy labels (LNL) is a challenge that arises in many real-world scenarios where collected training data can contain incorrect or corrupted labels. Most existing solutions identify noisy labels and adopt active learning to query human experts on them for denoising. In this paper, we propose an innovative collaborative learning framework NoiseAL based on active learning to combine large language models with small models for learning from noisy labels.
  arXiv  Detail & Related papers  (2025-04-03T04:36:39Z)
• Impact of Amplitude and Phase Damping Noise on Quantum Reinforcement Learning: Challenges and Opportunities [0.0]
  We investigate the effects of amplitude and phase damping noise on a quantum reinforcement learning algorithm. Our findings contribute to a deeper understanding of the role of noise in quantum learning algorithms.
  arXiv  Detail & Related papers  (2025-03-31T13:27:30Z)
• Enhancing Noise Robustness of Retrieval-Augmented Language Models with Adaptive Adversarial Training [39.21885486667879]
  Large Language Models (LLMs) exhibit substantial capabilities yet encounter challenges, including hallucination, outdated knowledge, and untraceable reasoning processes. Retrieval-augmented generation (RAG) has emerged as a promising solution, integrating knowledge from external databases to mitigate these challenges. We propose a novel RAG approach known as Retrieval-augmented Adaptive Adrial Training (RAAT)
  arXiv  Detail & Related papers  (2024-05-31T16:24:53Z)
• Enhancing Q-Learning with Large Language Model Heuristics [0.0]
  Large language models (LLMs) can achieve zero-shot learning for simpler tasks, but they suffer from low inference speeds and occasional hallucinations. We propose textbfLLM-guided Q-learning, a framework that leverages LLMs as hallucinations to aid in learning the Q-function for reinforcement learning.
  arXiv  Detail & Related papers  (2024-05-06T10:42:28Z)
• Deep Active Learning: A Reality Check [30.19086526296748]
  No single-model method decisively outperforms entropy-based active learning. We extend our evaluation to other tasks, exploring the active learning effectiveness in combination with semi-supervised learning.
  arXiv  Detail & Related papers  (2024-03-21T19:28:17Z)
• Transfer and Active Learning for Dissonance Detection: Addressing the Rare-Class Challenge [7.61140479230184]
  We propose and investigate transfer- and active learning solutions to the rare class problem of dissonance detection. We perform experiments for a specific rare class problem: collecting language samples of cognitive dissonance from social media. We find that probability-of-rare-class (PRC) is a simple and effective strategy to guide annotations and ultimately improve model accuracy.
  arXiv  Detail & Related papers  (2023-05-03T23:29:05Z)
• Responsible Active Learning via Human-in-the-loop Peer Study [88.01358655203441]
  We propose a responsible active learning method, namely Peer Study Learning (PSL), to simultaneously preserve data privacy and improve model stability. We first introduce a human-in-the-loop teacher-student architecture to isolate unlabelled data from the task learner (teacher) on the cloud-side. During training, the task learner instructs the light-weight active learner which then provides feedback on the active sampling criterion.
  arXiv  Detail & Related papers  (2022-11-24T13:18:27Z)
• What Makes Good Contrastive Learning on Small-Scale Wearable-based Tasks? [59.51457877578138]
  We study contrastive learning on the wearable-based activity recognition task. This paper presents an open-source PyTorch library textttCL-HAR, which can serve as a practical tool for researchers.
  arXiv  Detail & Related papers  (2022-02-12T06:10:15Z)
• Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answering [71.15403434929915]
  We show that across 5 models and 4 datasets on the task of visual question answering, a wide variety of active learning approaches fail to outperform random selection. We identify the problem as collective outliers -- groups of examples that active learning methods prefer to acquire but models fail to learn. We show that active learning sample efficiency increases significantly as the number of collective outliers in the active learning pool decreases.
  arXiv  Detail & Related papers  (2021-07-06T00:52:11Z)
• Rebuilding Trust in Active Learning with Actionable Metrics [77.99796068970569]
  Active Learning (AL) is an active domain of research, but is seldom used in the industry despite the pressing needs. This is in part due to a misalignment of objectives, while research strives at getting the best results on selected datasets. We present various actionable metrics to help rebuild trust of industrial practitioners in Active Learning.
  arXiv  Detail & Related papers  (2020-12-18T09:34:59Z)
• Bayesian active learning for production, a systematic study and a reusable library [85.32971950095742]
  In this paper, we analyse the main drawbacks of current active learning techniques. We do a systematic study on the effects of the most common issues of real-world datasets on the deep active learning process. We derive two techniques that can speed up the active learning loop such as partial uncertainty sampling and larger query size.
  arXiv  Detail & Related papers  (2020-06-17T14:51:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.