Heuristic Pathologies and Further Variance Reduction via Uncertainty Propagation in the AIVAT Family of Techniques

• URL: http://arxiv.org/abs/2605.14261v1
• Date: Thu, 14 May 2026 02:04:26 GMT
• Title: Heuristic Pathologies and Further Variance Reduction via Uncertainty Propagation in the AIVAT Family of Techniques
• Abstract summary: We show how the uncertainty can be propagated to quantify the uncertainty of AIVAT estimates.<n>In our experiments, we use a dataset of 10,000 poker hands to demonstrate our pathology and uncertainty results.
• Score: 62.94836578759359
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: How should an agent's performance in a multiagent environment be evaluated when there is a limited sample size or a high cost of running a trial? The AIVAT family of variance reduction techniques was proposed to address this challenge by introducing unbiased low-variance estimators of agents' expected payoffs. An important component of AIVAT is a heuristic value function that discriminates between potentially low- and high-value counterfactual histories. A notable gap in the literature is that there is little to no constraint or guideline on how the heuristic value function should be chosen or how uncertainty in its output should be handled. In our first contribution, we parameterize the heuristic value function to highlight AIVAT's potential vulnerabilities: a) the sample variance can be set pathologically low by directly applying gradient descent on the sample variance, and b) one can p-hack to draw a desired statistical conclusion via gradient descent/ascent on the test statistic. The main takeaway is that the heuristic value function should be fixed prior to observing the evaluation data! In our second contribution, we show how the heuristic uncertainty can be propagated to quantify the uncertainty of AIVAT estimates. It is then possible to further reduce the variance using inverse-variance weighted averaging, but AIVAT's unbiasedness guarantee may have to be sacrificed. In our experiments, we use a dataset of 10,000 poker hands to demonstrate our heuristic pathology and uncertainty results, with the latter yielding a 43.0% reduction in the number of samples (poker hands) needed to draw statistical conclusions.

Related papers

• Prediction-Powered Active Testing [14.083979525233316]
  Active testing provides a label--efficient approach to risk estimation by adaptively selecting which test points should be labelled.<n>We propose bfbfPrediction--Powered Active Testing (PPAT), a novel label--efficient risk estimation framework.
  arXiv  Detail & Related papers  (2026-07-09T10:52:24Z)
• Gradient boosting for extremes: sampling theory and application to insurance [0.0]
  We develop a theory for gradient boosting applied to the estimation of co-dependent Generalized Peaks of distributions.<n>We show that it significantly reduces gradient correlation during training and improves convergence stability.<n>The gradient boosting approach yields a good fit for the tail of settlement cost distributions and reveals that the number of days to settlement is the dominant predictor of tail heaviness.
  arXiv  Detail & Related papers  (2026-06-12T08:51:00Z)
• A Generative Framework for Causal Estimation via Importance-Weighted Diffusion Distillation [55.53426007439564]
  Estimating individualized treatment effects from observational data is a central challenge in causal inference.<n>In inverse probability weighting (IPW) is a well-established solution to this problem, but its integration into modern deep learning frameworks remains limited.<n>We propose Importance-Weighted Diffusion Distillation (IWDD), a novel generative framework that combines the pretraining of diffusion models with importance-weighted score distillation.
  arXiv  Detail & Related papers  (2025-05-16T17:00:52Z)
• Progressive Generalization Risk Reduction for Data-Efficient Causal Effect Estimation [30.49865329385806]
  Causal effect estimation (CEE) provides a crucial tool for predicting the unobserved counterfactual outcome for an entity. In this paper, we study a more realistic CEE setting where the labelled data samples are scarce at the beginning. We propose the Model Agnostic Causal Active Learning (MACAL) algorithm for batch-wise label acquisition.
  arXiv  Detail & Related papers  (2024-11-18T03:17:40Z)
• Prognostic Covariate Adjustment for Logistic Regression in Randomized Controlled Trials [1.5020330976600735]
  We show that prognostic score adjustment can increase the power of the Wald test for the conditional odds ratio under a fixed sample size. We utilize g-computation to expand the scope of prognostic score adjustment to inferences on the marginal risk difference, relative risk, and odds ratio estimands.
  arXiv  Detail & Related papers  (2024-02-29T06:53:16Z)
• A Stable and Efficient Covariate-Balancing Estimator for Causal Survival Effects [33.62020029799266]
  We address the problem of estimating survival causal effects in data with conditionally-independent censoring. This addresses the use of inverses of small estimated probabilities and the resulting amplification of estimation error. We validate our theoretical results in experiments on synthetic and semi-synthetic data.
  arXiv  Detail & Related papers  (2023-10-01T00:04:29Z)
• Selective Nonparametric Regression via Testing [54.20569354303575]
  We develop an abstention procedure via testing the hypothesis on the value of the conditional variance at a given point. Unlike existing methods, the proposed one allows to account not only for the value of the variance itself but also for the uncertainty of the corresponding variance predictor.
  arXiv  Detail & Related papers  (2023-09-28T13:04:11Z)
• U-Statistics for Importance-Weighted Variational Inference [29.750633016889655]
  We propose the use of U-statistics to reduce variance for estimation in importance-weighted variational inference. We find empirically that U-statistic variance reduction can lead to modest to significant improvements in inference performance on a range of models.
  arXiv  Detail & Related papers  (2023-02-27T16:08:43Z)
• Data-Driven Influence Functions for Optimization-Based Causal Inference [105.5385525290466]
  We study a constructive algorithm that approximates Gateaux derivatives for statistical functionals by finite differencing. We study the case where probability distributions are not known a priori but need to be estimated from data.
  arXiv  Detail & Related papers  (2022-08-29T16:16:22Z)
• Prediction Sets Adaptive to Unknown Covariate Shift [18.105704797438417]
  We show that prediction sets with finite-sample coverage guarantee are uninformative. We propose a novel flexible distribution-free method, PredSet-1Step, to efficiently construct prediction sets.
  arXiv  Detail & Related papers  (2022-03-11T17:53:14Z)
• Treatment Effect Risk: Bounds and Inference [58.442274475425144]
  Since the average treatment effect measures the change in social welfare, even if positive, there is a risk of negative effect on, say, some 10% of the population. In this paper we consider how to nonetheless assess this important risk measure, formalized as the conditional value at risk (CVaR) of the ITE distribution. Some bounds can also be interpreted as summarizing a complex CATE function into a single metric and are of interest independently of being a bound.
  arXiv  Detail & Related papers  (2022-01-15T17:21:26Z)
• Increasing the efficiency of randomized trial estimates via linear adjustment for a prognostic score [59.75318183140857]
  Estimating causal effects from randomized experiments is central to clinical research. Most methods for historical borrowing achieve reductions in variance by sacrificing strict type-I error rate control.
  arXiv  Detail & Related papers  (2020-12-17T21:10:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.