Learning to Rank for Active Learning via Multi-Task Bilevel Optimization
- URL: http://arxiv.org/abs/2310.17044v1
- Date: Wed, 25 Oct 2023 22:50:09 GMT
- Title: Learning to Rank for Active Learning via Multi-Task Bilevel Optimization
- Authors: Zixin Ding, Si Chen, Ruoxi Jia, Yuxin Chen
- Abstract summary: We propose a novel approach for active learning, which aims to select batches of unlabeled instances through a learned surrogate model for data acquisition.
A key challenge in this approach is developing an acquisition function that generalizes well, as the history of data, which forms part of the utility function's input, grows over time.
- Score: 29.207101107965563
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Active learning is a promising paradigm to reduce the labeling cost by
strategically requesting labels to improve model performance. However, existing
active learning methods often rely on expensive acquisition function to
compute, extensive modeling retraining and multiple rounds of interaction with
annotators. To address these limitations, we propose a novel approach for
active learning, which aims to select batches of unlabeled instances through a
learned surrogate model for data acquisition. A key challenge in this approach
is developing an acquisition function that generalizes well, as the history of
data, which forms part of the utility function's input, grows over time. Our
novel algorithmic contribution is a bilevel multi-task bilevel optimization
framework that predicts the relative utility -- measured by the validation
accuracy -- of different training sets, and ensures the learned acquisition
function generalizes effectively. For cases where validation accuracy is
expensive to evaluate, we introduce efficient interpolation-based surrogate
models to estimate the utility function, reducing the evaluation cost. We
demonstrate the performance of our approach through extensive experiments on
standard active classification benchmarks. By employing our learned utility
function, we show significant improvements over traditional techniques, paving
the way for more efficient and effective utility maximization in active
learning applications.
Related papers
- MALADY: Multiclass Active Learning with Auction Dynamics on Graphs [0.9831489366502301]
We introduce the Multiclass Active Learning with Auction Dynamics on Graphs (MALADY) framework for efficient active learning.
We generalize the auction dynamics algorithm on similarity graphs for semi-supervised learning in [24] to incorporate a more general optimization functional.
We also introduce a novel active learning acquisition function that uses the dual variable of the auction algorithm to measure the uncertainty in the classifier to prioritize queries near the decision boundaries between different classes.
arXiv Detail & Related papers (2024-09-14T16:20:26Z) - Learn from the Learnt: Source-Free Active Domain Adaptation via Contrastive Sampling and Visual Persistence [60.37934652213881]
Domain Adaptation (DA) facilitates knowledge transfer from a source domain to a related target domain.
This paper investigates a practical DA paradigm, namely Source data-Free Active Domain Adaptation (SFADA), where source data becomes inaccessible during adaptation.
We present learn from the learnt (LFTL), a novel paradigm for SFADA to leverage the learnt knowledge from the source pretrained model and actively iterated models without extra overhead.
arXiv Detail & Related papers (2024-07-26T17:51:58Z) - Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learning [55.96599486604344]
We introduce an approach aimed at enhancing the reasoning capabilities of Large Language Models (LLMs) through an iterative preference learning process.
We use Monte Carlo Tree Search (MCTS) to iteratively collect preference data, utilizing its look-ahead ability to break down instance-level rewards into more granular step-level signals.
The proposed algorithm employs Direct Preference Optimization (DPO) to update the LLM policy using this newly generated step-level preference data.
arXiv Detail & Related papers (2024-05-01T11:10:24Z) - Feature Alignment: Rethinking Efficient Active Learning via Proxy in the Context of Pre-trained Models [4.713652957384158]
Fine-tuning the pre-trained model with active learning holds promise for reducing annotation costs.
Recent research has proposed proxy-based active learning, which pre-computes features to reduce computational costs.
This approach often incurs a significant loss in active learning performance, sometimes outweighing the computational cost savings.
We show that suitable training methods can mitigate the decline of active learning performance caused by certain selection discrepancies.
arXiv Detail & Related papers (2024-03-02T06:01:34Z) - Compute-Efficient Active Learning [0.0]
Active learning aims at reducing labeling costs by selecting the most informative samples from an unlabeled dataset.
Traditional active learning process often demands extensive computational resources, hindering scalability and efficiency.
We present a novel method designed to alleviate the computational burden associated with active learning on massive datasets.
arXiv Detail & Related papers (2024-01-15T12:32:07Z) - An Experimental Design Framework for Label-Efficient Supervised Finetuning of Large Language Models [55.01592097059969]
Supervised finetuning on instruction datasets has played a crucial role in achieving the remarkable zero-shot generalization capabilities.
Active learning is effective in identifying useful subsets of samples to annotate from an unlabeled pool.
We propose using experimental design to circumvent the computational bottlenecks of active learning.
arXiv Detail & Related papers (2024-01-12T16:56:54Z) - SURF: Semi-supervised Reward Learning with Data Augmentation for
Feedback-efficient Preference-based Reinforcement Learning [168.89470249446023]
We present SURF, a semi-supervised reward learning framework that utilizes a large amount of unlabeled samples with data augmentation.
In order to leverage unlabeled samples for reward learning, we infer pseudo-labels of the unlabeled samples based on the confidence of the preference predictor.
Our experiments demonstrate that our approach significantly improves the feedback-efficiency of the preference-based method on a variety of locomotion and robotic manipulation tasks.
arXiv Detail & Related papers (2022-03-18T16:50:38Z) - One-Round Active Learning [13.25385227263705]
One-round active learning aims to select a subset of unlabeled data points that achieve the highest utility after being labeled.
We propose DULO, a general framework for one-round active learning based on the notion of data utility functions.
Our results demonstrate that while existing active learning approaches could succeed with multiple rounds, DULO consistently performs better in the one-round setting.
arXiv Detail & Related papers (2021-04-23T23:59:50Z) - Active Learning for Sequence Tagging with Deep Pre-trained Models and
Bayesian Uncertainty Estimates [52.164757178369804]
Recent advances in transfer learning for natural language processing in conjunction with active learning open the possibility to significantly reduce the necessary annotation budget.
We conduct an empirical study of various Bayesian uncertainty estimation methods and Monte Carlo dropout options for deep pre-trained models in the active learning framework.
We also demonstrate that to acquire instances during active learning, a full-size Transformer can be substituted with a distilled version, which yields better computational performance.
arXiv Detail & Related papers (2021-01-20T13:59:25Z) - Semi-supervised Batch Active Learning via Bilevel Optimization [89.37476066973336]
We formulate our approach as a data summarization problem via bilevel optimization.
We show that our method is highly effective in keyword detection tasks in the regime when only few labeled samples are available.
arXiv Detail & Related papers (2020-10-19T16:53:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.