Taming Small-sample Bias in Low-budget Active Learning
- URL: http://arxiv.org/abs/2306.11056v1
- Date: Mon, 19 Jun 2023 16:42:11 GMT
- Title: Taming Small-sample Bias in Low-budget Active Learning
- Authors: Linxin Song, Jieyu Zhang, Xiaotian Lu, Tianyi Zhou
- Abstract summary: Firth bias reduction can provably reduce the bias during the model training process but might hinder learning if its coefficient is not adaptive to the learning progress.
We propose curriculum Firth bias reduction (CHAIN) that can automatically adjust the coefficient to be adaptive to the training process.
- Score: 20.900107811622803
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Active learning (AL) aims to minimize the annotation cost by only querying a
few informative examples for each model training stage. However, training a
model on a few queried examples suffers from the small-sample bias. In this
paper, we address this small-sample bias issue in low-budget AL by exploring a
regularizer called Firth bias reduction, which can provably reduce the bias
during the model training process but might hinder learning if its coefficient
is not adaptive to the learning progress. Instead of tuning the coefficient for
each query round, which is sensitive and time-consuming, we propose the
curriculum Firth bias reduction (CHAIN) that can automatically adjust the
coefficient to be adaptive to the training process. Under both deep learning
and linear model settings, experiments on three benchmark datasets with several
widely used query strategies and hyperparameter searching methods show that
CHAIN can be used to build more efficient AL and can substantially improve the
progress made by each active learning query.
Related papers
- An Efficient Replay for Class-Incremental Learning with Pre-trained Models [0.0]
In class-incremental learning, the steady state among the weight guided by each class center is disrupted, which is significantly correlated with forgetting.
We propose a new method to overcoming forgetting.
arXiv Detail & Related papers (2024-08-15T11:26:28Z) - Rethinking Classifier Re-Training in Long-Tailed Recognition: A Simple
Logits Retargeting Approach [102.0769560460338]
We develop a simple logits approach (LORT) without the requirement of prior knowledge of the number of samples per class.
Our method achieves state-of-the-art performance on various imbalanced datasets, including CIFAR100-LT, ImageNet-LT, and iNaturalist 2018.
arXiv Detail & Related papers (2024-03-01T03:27:08Z) - Enhancing Consistency and Mitigating Bias: A Data Replay Approach for
Incremental Learning [100.7407460674153]
Deep learning systems are prone to catastrophic forgetting when learning from a sequence of tasks.
To mitigate the problem, a line of methods propose to replay the data of experienced tasks when learning new tasks.
However, it is not expected in practice considering the memory constraint or data privacy issue.
As a replacement, data-free data replay methods are proposed by inverting samples from the classification model.
arXiv Detail & Related papers (2024-01-12T12:51:12Z) - D-CALM: A Dynamic Clustering-based Active Learning Approach for
Mitigating Bias [13.008323851750442]
In this paper, we propose a novel adaptive clustering-based active learning algorithm, D-CALM, that dynamically adjusts clustering and annotation efforts.
Experiments on eight datasets for a diverse set of text classification tasks, including emotion, hatespeech, dialog act, and book type detection, demonstrate that our proposed algorithm significantly outperforms baseline AL approaches.
arXiv Detail & Related papers (2023-05-26T15:17:43Z) - Boosting Differentiable Causal Discovery via Adaptive Sample Reweighting [62.23057729112182]
Differentiable score-based causal discovery methods learn a directed acyclic graph from observational data.
We propose a model-agnostic framework to boost causal discovery performance by dynamically learning the adaptive weights for the Reweighted Score function, ReScore.
arXiv Detail & Related papers (2023-03-06T14:49:59Z) - Distributionally Robust Models with Parametric Likelihood Ratios [123.05074253513935]
Three simple ideas allow us to train models with DRO using a broader class of parametric likelihood ratios.
We find that models trained with the resulting parametric adversaries are consistently more robust to subpopulation shifts when compared to other DRO approaches.
arXiv Detail & Related papers (2022-04-13T12:43:12Z) - CMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep
Learning [55.733193075728096]
Modern deep neural networks can easily overfit to biased training data containing corrupted labels or class imbalance.
Sample re-weighting methods are popularly used to alleviate this data bias issue.
We propose a meta-model capable of adaptively learning an explicit weighting scheme directly from data.
arXiv Detail & Related papers (2022-02-11T13:49:51Z) - Optimizing Active Learning for Low Annotation Budgets [6.753808772846254]
In deep learning, active learning is usually implemented as an iterative process in which successive deep models are updated via fine tuning.
We tackle this issue by using an approach inspired by transfer learning.
We introduce a novel acquisition function which exploits the iterative nature of AL process to select samples in a more robust fashion.
arXiv Detail & Related papers (2022-01-18T18:53:10Z) - Prototypical Classifier for Robust Class-Imbalanced Learning [64.96088324684683]
We propose textitPrototypical, which does not require fitting additional parameters given the embedding network.
Prototypical produces balanced and comparable predictions for all classes even though the training set is class-imbalanced.
We test our method on CIFAR-10LT, CIFAR-100LT and Webvision datasets, observing that Prototypical obtains substaintial improvements compared with state of the arts.
arXiv Detail & Related papers (2021-10-22T01:55:01Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.