Revisiting Active Sequential Prediction-Powered Mean Estimation

• URL: http://arxiv.org/abs/2604.18569v1
• Date: Mon, 20 Apr 2026 17:55:17 GMT
• Title: Revisiting Active Sequential Prediction-Powered Mean Estimation
• Abstract summary: We develop a non-asymptotic analysis of the estimator and establish a data-dependent bound on its confidence interval.<n>Our analysis further suggests that when a no-regret learning approach is used to determine the query probability, the query probability converges to the constraint of the max value of the query probability.
• Score: 6.660458629649825
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work, we revisit the problem of active sequential prediction-powered mean estimation, where at each round one must decide the query probability of the ground-truth label upon observing the covariates of a sample. Furthermore, if the label is not queried, the prediction from a machine learning model is used instead. Prior work proposed an elegant scheme that determines the query probability by combining an uncertainty-based suggestion with a constant probability that encodes a soft constraint on the query probability. We explored different values of the mixing parameter and observed an intriguing empirical pattern: the smallest confidence width tends to occur when the weight on the constant probability is close to one, thereby reducing the influence of the uncertainty-based component. Motivated by this observation, we develop a non-asymptotic analysis of the estimator and establish a data-dependent bound on its confidence interval. Our analysis further suggests that when a no-regret learning approach is used to determine the query probability and control this bound, the query probability converges to the constraint of the max value of the query probability when it is chosen obliviously to the current covariates. We also conduct simulations that corroborate these theoretical findings.

Related papers

• Uncertainty Estimation using Variance-Gated Distributions [0.6340400318304492]
  We propose an intuitive framework for uncertainty estimation and decomposition based on the signal-to-noise ratio of class probability distributions.<n>We introduce a variance-gated measure that scales predictions by a confidence factor derived from ensembles.
  arXiv  Detail & Related papers  (2025-09-07T16:19:21Z)
• Model uncertainty quantification using feature confidence sets for outcome excursions [0.0]
  This paper introduces a novel, model-agnostic framework for quantifying uncertainty in continuous and binary outcomes.<n>It is validated through simulations and applied to real-world datasets in contexts such as housing price prediction and time to sepsis diagnosis in healthcare.
  arXiv  Detail & Related papers  (2025-04-28T04:08:07Z)
• Bayesian meta learning for trustworthy uncertainty quantification [3.683202928838613]
  We propose, Trust-Bayes, a novel optimization framework for Bayesian meta learning. We characterize the lower bounds of the probabilities of the ground truth being captured by the specified intervals. We analyze the sample complexity with respect to the feasible probability for trustworthy uncertainty quantification.
  arXiv  Detail & Related papers  (2024-07-27T15:56:12Z)
• Score Matching-based Pseudolikelihood Estimation of Neural Marked Spatio-Temporal Point Process with Uncertainty Quantification [59.81904428056924]
  We introduce SMASH: a Score MAtching estimator for learning markedPs with uncertainty quantification. Specifically, our framework adopts a normalization-free objective by estimating the pseudolikelihood of markedPs through score-matching. The superior performance of our proposed framework is demonstrated through extensive experiments in both event prediction and uncertainty quantification.
  arXiv  Detail & Related papers  (2023-10-25T02:37:51Z)
• Invariant Probabilistic Prediction [45.90606906307022]
  We show that arbitrary distribution shifts do not, in general, admit invariant and robust probabilistic predictions. We propose a method to yield invariant probabilistic predictions, called IPP, and study the consistency of the underlying parameters.
  arXiv  Detail & Related papers  (2023-09-18T18:50:24Z)
• The Implicit Delta Method [61.36121543728134]
  In this paper, we propose an alternative, the implicit delta method, which works by infinitesimally regularizing the training loss of uncertainty. We show that the change in the evaluation due to regularization is consistent for the variance of the evaluation estimator, even when the infinitesimal change is approximated by a finite difference.
  arXiv  Detail & Related papers  (2022-11-11T19:34:17Z)
• Uncertainty estimation of pedestrian future trajectory using Bayesian approximation [137.00426219455116]
  Under dynamic traffic scenarios, planning based on deterministic predictions is not trustworthy. The authors propose to quantify uncertainty during forecasting using approximation which deterministic approaches fail to capture. The effect of dropout weights and long-term prediction on future state uncertainty has been studied.
  arXiv  Detail & Related papers  (2022-05-04T04:23:38Z)
• Dense Uncertainty Estimation via an Ensemble-based Conditional Latent Variable Model [68.34559610536614]
  We argue that the aleatoric uncertainty is an inherent attribute of the data and can only be correctly estimated with an unbiased oracle model. We propose a new sampling and selection strategy at train time to approximate the oracle model for aleatoric uncertainty estimation. Our results show that our solution achieves both accurate deterministic results and reliable uncertainty estimation.
  arXiv  Detail & Related papers  (2021-11-22T08:54:10Z)
• Dense Uncertainty Estimation [62.23555922631451]
  In this paper, we investigate neural networks and uncertainty estimation techniques to achieve both accurate deterministic prediction and reliable uncertainty estimation. We work on two types of uncertainty estimations solutions, namely ensemble based methods and generative model based methods, and explain their pros and cons while using them in fully/semi/weakly-supervised framework.
  arXiv  Detail & Related papers  (2021-10-13T01:23:48Z)
• The Hidden Uncertainty in a Neural Networks Activations [105.4223982696279]
  The distribution of a neural network's latent representations has been successfully used to detect out-of-distribution (OOD) data. This work investigates whether this distribution correlates with a model's epistemic uncertainty, thus indicating its ability to generalise to novel inputs.
  arXiv  Detail & Related papers  (2020-12-05T17:30:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.