Entropic Open-set Active Learning
- URL: http://arxiv.org/abs/2312.14126v1
- Date: Thu, 21 Dec 2023 18:47:12 GMT
- Title: Entropic Open-set Active Learning
- Authors: Bardia Safaei, Vibashan VS, Celso M. de Melo, Vishal M. Patel
- Abstract summary: Active Learning (AL) aims to enhance the performance of deep models by selecting the most informative samples for annotation from a pool of unlabeled data.
Despite impressive performance in closed-set settings, most AL methods fail in real-world scenarios where the unlabeled data contains unknown categories.
We propose an Entropic Open-set AL framework which leverages both known and unknown distributions effectively to select informative samples during AL rounds.
- Score: 30.91182106918793
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Active Learning (AL) aims to enhance the performance of deep models by
selecting the most informative samples for annotation from a pool of unlabeled
data. Despite impressive performance in closed-set settings, most AL methods
fail in real-world scenarios where the unlabeled data contains unknown
categories. Recently, a few studies have attempted to tackle the AL problem for
the open-set setting. However, these methods focus more on selecting known
samples and do not efficiently utilize unknown samples obtained during AL
rounds. In this work, we propose an Entropic Open-set AL (EOAL) framework which
leverages both known and unknown distributions effectively to select
informative samples during AL rounds. Specifically, our approach employs two
different entropy scores. One measures the uncertainty of a sample with respect
to the known-class distributions. The other measures the uncertainty of the
sample with respect to the unknown-class distributions. By utilizing these two
entropy scores we effectively separate the known and unknown samples from the
unlabeled data resulting in better sampling. Through extensive experiments, we
show that the proposed method outperforms existing state-of-the-art methods on
CIFAR-10, CIFAR-100, and TinyImageNet datasets. Code is available at
\url{https://github.com/bardisafa/EOAL}.
Related papers
- Dirichlet-Based Coarse-to-Fine Example Selection For Open-Set Annotation [37.33424244520009]
We propose a Dirichlet-based Coarse-to-Fine Example Selection (DCFS) strategy.
Our method introduces simplex-based evidential deep learning (EDL) to break translation invariance.
Experiments on various openness ratio datasets demonstrate that DCFS achieves state-of-art performance.
arXiv Detail & Related papers (2024-09-26T07:47:50Z) - Bidirectional Uncertainty-Based Active Learning for Open Set Annotation [28.624321743629825]
Active learning in open set scenarios presents a novel challenge of identifying the most valuable examples.
Traditional methods prioritize selecting informative examples with low confidence.
Recent methods favor the most probable known-class examples.
We propose a Bidirectional Uncertainty-based Active Learning framework.
arXiv Detail & Related papers (2024-02-23T08:59:04Z) - MyriadAL: Active Few Shot Learning for Histopathology [10.652626309100889]
We introduce an active few shot learning framework, Myriad Active Learning (MAL)
MAL includes a contrastive-learning encoder, pseudo-label generation, and novel query sample selection in the loop.
Experiments on two public histopathology datasets show that MAL has superior test accuracy, macro F1-score, and label efficiency compared to prior works.
arXiv Detail & Related papers (2023-10-24T20:08:15Z) - Adaptive Negative Evidential Deep Learning for Open-set Semi-supervised Learning [69.81438976273866]
Open-set semi-supervised learning (Open-set SSL) considers a more practical scenario, where unlabeled data and test data contain new categories (outliers) not observed in labeled data (inliers)
We introduce evidential deep learning (EDL) as an outlier detector to quantify different types of uncertainty, and design different uncertainty metrics for self-training and inference.
We propose a novel adaptive negative optimization strategy, making EDL more tailored to the unlabeled dataset containing both inliers and outliers.
arXiv Detail & Related papers (2023-03-21T09:07:15Z) - Prompt-driven efficient Open-set Semi-supervised Learning [52.30303262499391]
Open-set semi-supervised learning (OSSL) has attracted growing interest, which investigates a more practical scenario where out-of-distribution (OOD) samples are only contained in unlabeled data.
We propose a prompt-driven efficient OSSL framework, called OpenPrompt, which can propagate class information from labeled to unlabeled data with only a small number of trainable parameters.
arXiv Detail & Related papers (2022-09-28T16:25:08Z) - Pareto Optimization for Active Learning under Out-of-Distribution Data
Scenarios [79.02009938011447]
We propose a sampling scheme, which selects optimal subsets of unlabeled samples with fixed batch size from the unlabeled data pool.
Experimental results show its effectiveness on both classical Machine Learning (ML) and Deep Learning (DL) tasks.
arXiv Detail & Related papers (2022-07-04T04:11:44Z) - UNICON: Combating Label Noise Through Uniform Selection and Contrastive
Learning [89.56465237941013]
We propose UNICON, a simple yet effective sample selection method which is robust to high label noise.
We obtain an 11.4% improvement over the current state-of-the-art on CIFAR100 dataset with a 90% noise rate.
arXiv Detail & Related papers (2022-03-28T07:36:36Z) - Active Learning by Feature Mixing [52.16150629234465]
We propose a novel method for batch active learning called ALFA-Mix.
We identify unlabelled instances with sufficiently-distinct features by seeking inconsistencies in predictions.
We show that inconsistencies in these predictions help discovering features that the model is unable to recognise in the unlabelled instances.
arXiv Detail & Related papers (2022-03-14T12:20:54Z) - Minimax Active Learning [61.729667575374606]
Active learning aims to develop label-efficient algorithms by querying the most representative samples to be labeled by a human annotator.
Current active learning techniques either rely on model uncertainty to select the most uncertain samples or use clustering or reconstruction to choose the most diverse set of unlabeled examples.
We develop a semi-supervised minimax entropy-based active learning algorithm that leverages both uncertainty and diversity in an adversarial manner.
arXiv Detail & Related papers (2020-12-18T19:03:40Z) - Deep Active Learning via Open Set Recognition [0.0]
In many applications, data is easy to acquire but expensive and time-consuming to label prominent examples.
We formulate active learning as an open-set recognition problem.
Unlike current active learning methods, our algorithm can learn tasks without the need for task labels.
arXiv Detail & Related papers (2020-07-04T22:09:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.