A Planning Framework for Adaptive Labeling

• URL: http://arxiv.org/abs/2502.06076v1
• Date: Mon, 10 Feb 2025 00:01:08 GMT
• Title: A Planning Framework for Adaptive Labeling
• Authors: Daksh Mittal, Yuanzhe Ma, Shalmali Joshi, Hongseok Namkoong,
• Abstract summary: We introduce an adaptive labeling framework where measurement effort can be reallocated in batches. We show that even one-step lookahead policy can substantially outperform common adaptive labelings. We propose a direct backpropagation-based approach, Smoothed-Autodiff, based on a carefully smoothed version of the original non-differentiable MDP.
• Score: 8.883000217198843
• License:
• Abstract: Ground truth labels/outcomes are critical for advancing scientific and engineering applications, e.g., evaluating the treatment effect of an intervention or performance of a predictive model. Since randomly sampling inputs for labeling can be prohibitively expensive, we introduce an adaptive labeling framework where measurement effort can be reallocated in batches. We formulate this problem as a Markov decision process where posterior beliefs evolve over time as batches of labels are collected (state transition), and batches (actions) are chosen to minimize uncertainty at the end of data collection. We design a computational framework that is agnostic to different uncertainty quantification approaches including those based on deep learning, and allows a diverse array of policy gradient approaches by relying on continuous policy parameterizations. On real and synthetic datasets, we demonstrate even a one-step lookahead policy can substantially outperform common adaptive labeling heuristics, highlighting the virtue of planning. On the methodological side, we note that standard REINFORCE-style policy gradient estimators can suffer high variance since they rely only on zeroth order information. We propose a direct backpropagation-based approach, Smoothed-Autodiff, based on a carefully smoothed version of the original non-differentiable MDP. Our method enjoys low variance at the price of introducing bias, and we theoretically and empirically show that this trade-off can be favorable.

Related papers

• Stratified Prediction-Powered Inference for Hybrid Language Model Evaluation [62.2436697657307]
  Prediction-powered inference (PPI) is a method that improves statistical estimates based on limited human-labeled data. We propose a method called Stratified Prediction-Powered Inference (StratPPI) We show that the basic PPI estimates can be considerably improved by employing simple data stratification strategies.
  arXiv  Detail & Related papers  (2024-06-06T17:37:39Z)
• Offline Bayesian Aleatoric and Epistemic Uncertainty Quantification and Posterior Value Optimisation in Finite-State MDPs [3.1139806580181006]
  We address the challenge of quantifying Bayesian uncertainty in offline use cases of finite-state Markov Decision Processes (MDPs) with unknown dynamics. We use standard Bayesian reinforcement learning methods to capture the posterior uncertainty in MDP parameters. We then analytically compute the first two moments of the return distribution across posterior samples and apply the law of total variance. We highlight the real-world impact and computational scalability of our method by applying it to the AI Clinician problem.
  arXiv  Detail & Related papers  (2024-06-04T16:21:14Z)
• Optimal Baseline Corrections for Off-Policy Contextual Bandits [61.740094604552475]
  We aim to learn decision policies that optimize an unbiased offline estimate of an online reward metric. We propose a single framework built on their equivalence in learning scenarios. Our framework enables us to characterize the variance-optimal unbiased estimator and provide a closed-form solution for it.
  arXiv  Detail & Related papers  (2024-05-09T12:52:22Z)
• Dirichlet-Based Prediction Calibration for Learning with Noisy Labels [40.78497779769083]
  Learning with noisy labels can significantly hinder the generalization performance of deep neural networks (DNNs) Existing approaches address this issue through loss correction or example selection methods. We propose the textitDirichlet-based Prediction (DPC) method as a solution.
  arXiv  Detail & Related papers  (2024-01-13T12:33:04Z)
• Hypothesis Testing for Class-Conditional Noise Using Local Maximum Likelihood [1.8798171797988192]
  In supervised learning, automatically assessing the quality of the labels before any learning takes place remains an open research question. In this paper we show how similar procedures can be followed when the underlying model is a product of Local Maximum Likelihood Estimation. This different view allows for wider applicability of the tests by offering users access to a richer model class.
  arXiv  Detail & Related papers  (2023-12-15T22:14:58Z)
• Off-Policy Evaluation for Large Action Spaces via Policy Convolution [60.6953713877886]
  Policy Convolution family of estimators uses latent structure within actions to strategically convolve the logging and target policies. Experiments on synthetic and benchmark datasets demonstrate remarkable mean squared error (MSE) improvements when using PC.
  arXiv  Detail & Related papers  (2023-10-24T01:00:01Z)
• Partial-Label Regression [54.74984751371617]
  Partial-label learning is a weakly supervised learning setting that allows each training example to be annotated with a set of candidate labels. Previous studies on partial-label learning only focused on the classification setting where candidate labels are all discrete. In this paper, we provide the first attempt to investigate partial-label regression, where each training example is annotated with a set of real-valued candidate labels.
  arXiv  Detail & Related papers  (2023-06-15T09:02:24Z)
• Partial sequence labeling with structured Gaussian Processes [8.239028141030621]
  We propose structured Gaussian Processes for partial sequence labeling. It encodes uncertainty in the prediction and does not need extra effort for model selection and hyper parameter learning. It is evaluated on several sequence labeling tasks and the experimental results show the effectiveness of the proposed model.
  arXiv  Detail & Related papers  (2022-09-20T00:56:49Z)
• Don't Throw it Away! The Utility of Unlabeled Data in Fair Decision Making [14.905698014932488]
  We propose a novel method based on a variational autoencoder for practical fair decision-making. Our method learns an unbiased data representation leveraging both labeled and unlabeled data. Our method converges to the optimal (fair) policy according to the ground-truth with low variance.
  arXiv  Detail & Related papers  (2022-05-10T10:33:11Z)
• Delving into Probabilistic Uncertainty for Unsupervised Domain Adaptive Person Re-Identification [54.174146346387204]
  We propose an approach named probabilistic uncertainty guided progressive label refinery (P$2$LR) for domain adaptive person re-identification. A quantitative criterion is established to measure the uncertainty of pseudo labels and facilitate the network training. Our method outperforms the baseline by 6.5% mAP on the Duke2Market task, while surpassing the state-of-the-art method by 2.5% mAP on the Market2MSMT task.
  arXiv  Detail & Related papers  (2021-12-28T07:40:12Z)
• Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning [78.83598532168256]
  Marginal-likelihood based model-selection is rarely used in deep learning due to estimation difficulties. Our work shows that marginal likelihoods can improve generalization and be useful when validation data is unavailable.
  arXiv  Detail & Related papers  (2021-04-11T09:50:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.