Demistifying Inference after Adaptive Experiments

• URL: http://arxiv.org/abs/2405.01281v1
• Date: Thu, 2 May 2024 13:39:51 GMT
• Title: Demistifying Inference after Adaptive Experiments
• Authors: Aurélien Bibaut, Nathan Kallus,
• Abstract summary: Adaptive experiments such as multi-arm bandits adapt the treatment-allocation policy and/or the decision to stop the experiment to the data observed so far. The concentration inequalities and union bounds that generally underlie adaptive experimentation algorithms can yield overly conservative inferences. In this article we aim to explain why, how, and when adaptivity is in fact an issue for inference and, when it is, understand the various ways to fix it.
• Score: 43.653628046172656
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Adaptive experiments such as multi-arm bandits adapt the treatment-allocation policy and/or the decision to stop the experiment to the data observed so far. This has the potential to improve outcomes for study participants within the experiment, to improve the chance of identifying best treatments after the experiment, and to avoid wasting data. Seen as an experiment (rather than just a continually optimizing system) it is still desirable to draw statistical inferences with frequentist guarantees. The concentration inequalities and union bounds that generally underlie adaptive experimentation algorithms can yield overly conservative inferences, but at the same time the asymptotic normality we would usually appeal to in non-adaptive settings can be imperiled by adaptivity. In this article we aim to explain why, how, and when adaptivity is in fact an issue for inference and, when it is, understand the various ways to fix it: reweighting to stabilize variances and recover asymptotic normality, always-valid inference based on joint normality of an asymptotic limiting sequence, and characterizing and inverting the non-normal distributions induced by adaptivity.

Related papers

• Inference for Batched Adaptive Experiments [0.0]
  This note suggests a BOLS test statistic for inference of treatment effects in adaptive experiments.<n>We provide simulation results comparing rejection rates in the typical case with few treatment periods and few (or many) observations per batch.
  arXiv  Detail & Related papers  (2025-12-10T23:33:08Z)
• Minimax and Bayes Optimal Adaptive Experimental Design for Treatment Choice [6.44705221140412]
  We consider an adaptive experiment for treatment choice and design a minimax and Bayes optimal adaptive experiment with respect to regret.<n>We show that this experiment, often referred to as Neyman allocation, is minimax and Bayes optimal in the sense that its regret upper bounds exactly match the lower bounds that we derive.
  arXiv  Detail & Related papers  (2025-12-09T11:58:27Z)
• Kernel Treatment Effects with Adaptively Collected Data [23.3862001690226]
  We present the first kernel-based inference framework for distributional inference under adaptive data collection.<n>Our method combines doubly robust scores with variance stabilization to ensure normality via a Hilbert-space martingale CLT.<n>Experiments show it is well and effective for both mean shifts and higher-moment differences.
  arXiv  Detail & Related papers  (2025-10-11T15:01:21Z)
• Adaptive Experimentation When You Can't Experiment [55.86593195947978]
  This paper introduces the emphconfounded pure exploration transductive linear bandit (textttCPET-LB) problem. Online services can employ a properly randomized encouragement that incentivizes users toward a specific treatment.
  arXiv  Detail & Related papers  (2024-06-15T20:54:48Z)
• Mitigating LLM Hallucinations via Conformal Abstention [70.83870602967625]
  We develop a principled procedure for determining when a large language model should abstain from responding in a general domain. We leverage conformal prediction techniques to develop an abstention procedure that benefits from rigorous theoretical guarantees on the hallucination rate (error rate) Experimentally, our resulting conformal abstention method reliably bounds the hallucination rate on various closed-book, open-domain generative question answering datasets.
  arXiv  Detail & Related papers  (2024-04-04T11:32:03Z)
• Optimal Ridge Regularization for Out-of-Distribution Prediction [6.278498348219108]
  We study the behavior of optimal ridge regularization and optimal ridge risk for out-of-distribution prediction. We establish general conditions that determine the sign of the optimal regularization level.
  arXiv  Detail & Related papers  (2024-04-01T16:51:19Z)
• Semiparametric Efficient Inference in Adaptive Experiments [29.43493007296859]
  We consider the problem of efficient inference of the Average Treatment Effect in a sequential experiment where the policy governing the assignment of subjects to treatment or control can change over time. We first provide a central limit theorem for the Adaptive Augmented Inverse-Probability Weighted estimator, which is semi efficient, under weaker assumptions than those previously made in the literature. We then consider sequential inference setting, deriving both propensity and nonasymptotic confidence sequences that are considerably tighter than previous methods.
  arXiv  Detail & Related papers  (2023-11-30T06:25:06Z)
• Optimal Conditional Inference in Adaptive Experiments [1.8130068086063336]
  We consider the problem of conditional inference on the realized stopping time, assignment probabilities, and target parameter, where all of these may be chosen adaptively using information up to the last batch of the experiment. Absent further restrictions on the experiment, we show that inference using only the results of the last batch is optimal.
  arXiv  Detail & Related papers  (2023-09-21T15:17:38Z)
• DELTA: degradation-free fully test-time adaptation [59.74287982885375]
  We find that two unfavorable defects are concealed in the prevalent adaptation methodologies like test-time batch normalization (BN) and self-learning. First, we reveal that the normalization statistics in test-time BN are completely affected by the currently received test samples, resulting in inaccurate estimates. Second, we show that during test-time adaptation, the parameter update is biased towards some dominant classes.
  arXiv  Detail & Related papers  (2023-01-30T15:54:00Z)
• Near-optimal inference in adaptive linear regression [60.08422051718195]
  Even simple methods like least squares can exhibit non-normal behavior when data is collected in an adaptive manner. We propose a family of online debiasing estimators to correct these distributional anomalies in at least squares estimation. We demonstrate the usefulness of our theory via applications to multi-armed bandit, autoregressive time series estimation, and active learning with exploration.
  arXiv  Detail & Related papers  (2021-07-05T21:05:11Z)
• Post-Contextual-Bandit Inference [57.88785630755165]
  Contextual bandit algorithms are increasingly replacing non-adaptive A/B tests in e-commerce, healthcare, and policymaking. They can both improve outcomes for study participants and increase the chance of identifying good or even best policies. To support credible inference on novel interventions at the end of the study, we still want to construct valid confidence intervals on average treatment effects, subgroup effects, or value of new policies.
  arXiv  Detail & Related papers  (2021-06-01T12:01:51Z)
• Counterfactual Inference of the Mean Outcome under a Convergence of Average Logging Probability [5.596752018167751]
  This paper considers estimating the mean outcome of an action from samples obtained in adaptive experiments. In adaptive experiments, the probability of choosing an action is allowed to be sequentially updated based on past observations.
  arXiv  Detail & Related papers  (2021-02-17T19:05:53Z)
• Conformal Inference of Counterfactuals and Individual Treatment Effects [6.810856082577402]
  We propose a conformal inference-based approach that can produce reliable interval estimates for counterfactuals and individual treatment effects. Existing methods suffer from a significant coverage deficit even in simple models.
  arXiv  Detail & Related papers  (2020-06-11T01:03:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.