Learning to Rank for Active Learning via Multi-Task Bilevel Optimization
- URL: http://arxiv.org/abs/2310.17044v1
- Date: Wed, 25 Oct 2023 22:50:09 GMT
- Title: Learning to Rank for Active Learning via Multi-Task Bilevel Optimization
- Authors: Zixin Ding, Si Chen, Ruoxi Jia, Yuxin Chen
- Abstract summary: We propose a novel approach for active learning, which aims to select batches of unlabeled instances through a learned surrogate model for data acquisition.
A key challenge in this approach is developing an acquisition function that generalizes well, as the history of data, which forms part of the utility function's input, grows over time.
- Score: 29.207101107965563
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Active learning is a promising paradigm to reduce the labeling cost by
strategically requesting labels to improve model performance. However, existing
active learning methods often rely on expensive acquisition function to
compute, extensive modeling retraining and multiple rounds of interaction with
annotators. To address these limitations, we propose a novel approach for
active learning, which aims to select batches of unlabeled instances through a
learned surrogate model for data acquisition. A key challenge in this approach
is developing an acquisition function that generalizes well, as the history of
data, which forms part of the utility function's input, grows over time. Our
novel algorithmic contribution is a bilevel multi-task bilevel optimization
framework that predicts the relative utility -- measured by the validation
accuracy -- of different training sets, and ensures the learned acquisition
function generalizes effectively. For cases where validation accuracy is
expensive to evaluate, we introduce efficient interpolation-based surrogate
models to estimate the utility function, reducing the evaluation cost. We
demonstrate the performance of our approach through extensive experiments on
standard active classification benchmarks. By employing our learned utility
function, we show significant improvements over traditional techniques, paving
the way for more efficient and effective utility maximization in active
learning applications.
Related papers
- Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learning [55.96599486604344]
We introduce an approach aimed at enhancing the reasoning capabilities of Large Language Models (LLMs) through an iterative preference learning process.
We use Monte Carlo Tree Search (MCTS) to iteratively collect preference data, utilizing its look-ahead ability to break down instance-level rewards into more granular step-level signals.
The proposed algorithm employs Direct Preference Optimization (DPO) to update the LLM policy using this newly generated step-level preference data.
arXiv Detail & Related papers (2024-05-01T11:10:24Z) - Compute-Efficient Active Learning [0.0]
Active learning aims at reducing labeling costs by selecting the most informative samples from an unlabeled dataset.
Traditional active learning process often demands extensive computational resources, hindering scalability and efficiency.
We present a novel method designed to alleviate the computational burden associated with active learning on massive datasets.
arXiv Detail & Related papers (2024-01-15T12:32:07Z) - An Experimental Design Framework for Label-Efficient Supervised Finetuning of Large Language Models [55.01592097059969]
Supervised finetuning on instruction datasets has played a crucial role in achieving the remarkable zero-shot generalization capabilities.
Active learning is effective in identifying useful subsets of samples to annotate from an unlabeled pool.
We propose using experimental design to circumvent the computational bottlenecks of active learning.
arXiv Detail & Related papers (2024-01-12T16:56:54Z) - Perturbation-based Active Learning for Question Answering [25.379528163789082]
Building a question answering (QA) model with less annotation costs can be achieved by utilizing active learning (AL) training strategy.
It selects the most informative unlabeled training data to update the model effectively.
arXiv Detail & Related papers (2023-11-04T08:07:23Z) - MeanAP-Guided Reinforced Active Learning for Object Detection [34.19741444116433]
This paper introduces MeanAP-Guided Reinforced Active Learning for Object Detection (MAGRAL)
Built upon LSTM architecture, the agent efficiently explores and selects subsequent training instances.
We assess MAGRAL's efficacy across popular benchmarks, PASCAL VOC and MS COCO.
arXiv Detail & Related papers (2023-10-12T14:59:22Z) - HyperImpute: Generalized Iterative Imputation with Automatic Model
Selection [77.86861638371926]
We propose a generalized iterative imputation framework for adaptively and automatically configuring column-wise models.
We provide a concrete implementation with out-of-the-box learners, simulators, and interfaces.
arXiv Detail & Related papers (2022-06-15T19:10:35Z) - SURF: Semi-supervised Reward Learning with Data Augmentation for
Feedback-efficient Preference-based Reinforcement Learning [168.89470249446023]
We present SURF, a semi-supervised reward learning framework that utilizes a large amount of unlabeled samples with data augmentation.
In order to leverage unlabeled samples for reward learning, we infer pseudo-labels of the unlabeled samples based on the confidence of the preference predictor.
Our experiments demonstrate that our approach significantly improves the feedback-efficiency of the preference-based method on a variety of locomotion and robotic manipulation tasks.
arXiv Detail & Related papers (2022-03-18T16:50:38Z) - One-Round Active Learning [13.25385227263705]
One-round active learning aims to select a subset of unlabeled data points that achieve the highest utility after being labeled.
We propose DULO, a general framework for one-round active learning based on the notion of data utility functions.
Our results demonstrate that while existing active learning approaches could succeed with multiple rounds, DULO consistently performs better in the one-round setting.
arXiv Detail & Related papers (2021-04-23T23:59:50Z) - Active Learning for Sequence Tagging with Deep Pre-trained Models and
Bayesian Uncertainty Estimates [52.164757178369804]
Recent advances in transfer learning for natural language processing in conjunction with active learning open the possibility to significantly reduce the necessary annotation budget.
We conduct an empirical study of various Bayesian uncertainty estimation methods and Monte Carlo dropout options for deep pre-trained models in the active learning framework.
We also demonstrate that to acquire instances during active learning, a full-size Transformer can be substituted with a distilled version, which yields better computational performance.
arXiv Detail & Related papers (2021-01-20T13:59:25Z) - Semi-supervised Batch Active Learning via Bilevel Optimization [89.37476066973336]
We formulate our approach as a data summarization problem via bilevel optimization.
We show that our method is highly effective in keyword detection tasks in the regime when only few labeled samples are available.
arXiv Detail & Related papers (2020-10-19T16:53:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.