Active Adaptive Experimental Design for Treatment Effect Estimation with Covariate Choices

• URL: http://arxiv.org/abs/2403.03589v2
• Date: Tue, 18 Jun 2024 18:20:08 GMT
• Title: Active Adaptive Experimental Design for Treatment Effect Estimation with Covariate Choices
• Authors: Masahiro Kato, Akihiro Oga, Wataru Komatsubara, Ryo Inokuchi,
• Abstract summary: This study designs an adaptive experiment for efficiently estimating average treatment effects (ATEs) In each round of our adaptive experiment, an experimenter samples an experimental unit, assigns a treatment, and observes the corresponding outcome immediately. At the end of the experiment, the experimenter estimates an ATE using the gathered samples.
• Score: 7.21848268647674
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This study designs an adaptive experiment for efficiently estimating average treatment effects (ATEs). In each round of our adaptive experiment, an experimenter sequentially samples an experimental unit, assigns a treatment, and observes the corresponding outcome immediately. At the end of the experiment, the experimenter estimates an ATE using the gathered samples. The objective is to estimate the ATE with a smaller asymptotic variance. Existing studies have designed experiments that adaptively optimize the propensity score (treatment-assignment probability). As a generalization of such an approach, we propose optimizing the covariate density as well as the propensity score. First, we derive the efficient covariate density and propensity score that minimize the semiparametric efficiency bound and find that optimizing both covariate density and propensity score minimizes the semiparametric efficiency bound more effectively than optimizing only the propensity score. Next, we design an adaptive experiment using the efficient covariate density and propensity score sequentially estimated during the experiment. Lastly, we propose an ATE estimator whose asymptotic variance aligns with the minimized semiparametric efficiency bound.

Related papers

• Prediction-Guided Active Experiments [18.494123886098215]
  We introduce a new framework for active experimentation, the Prediction-Guided Active Experiment (PGAE) PGAE leverages predictions from an existing machine learning model to guide sampling and experimentation. We show that PGAE remains efficient and attains the same semi-parametric bound under certain regularity assumptions.
  arXiv  Detail & Related papers  (2024-11-18T20:16:24Z)
• Variance reduction combining pre-experiment and in-experiment data [0.0]
  Online controlled experiments (A/B testing) are essential in data-driven decision-making for many companies. Existing methods like CUPED and CUPAC use pre-experiment data to reduce variance, but their effectiveness depends on the correlation between the pre-experiment data and the outcome. We introduce a novel method that combines both pre-experiment and in-experiment data to achieve greater variance reduction than CUPED and CUPAC.
  arXiv  Detail & Related papers  (2024-10-11T17:45:29Z)
• STATE: A Robust ATE Estimator of Heavy-Tailed Metrics for Variance Reduction in Online Controlled Experiments [22.32661807469984]
  We develop a novel framework that integrates the Student's t-distribution with machine learning tools to fit heavy-tailed metrics. By adopting a variational EM method to optimize the loglikehood function, we can infer a robust solution that greatly eliminates the negative impact of outliers. Both simulations on synthetic data and long-term empirical results on Meituan experiment platform demonstrate the effectiveness of our method.
  arXiv  Detail & Related papers  (2024-07-23T09:35:59Z)
• Efficient adjustment for complex covariates: Gaining efficiency with DOPE [56.537164957672715]
  We propose a framework that accommodates adjustment for any subset of information expressed by the covariates. Based on our theoretical results, we propose the Debiased Outcome-adapted Propensity Estorimator (DOPE) for efficient estimation of the average treatment effect (ATE) Our results show that the DOPE provides an efficient and robust methodology for ATE estimation in various observational settings.
  arXiv  Detail & Related papers  (2024-02-20T13:02:51Z)
• Adaptive Instrument Design for Indirect Experiments [48.815194906471405]
  Unlike RCTs, indirect experiments estimate treatment effects by leveragingconditional instrumental variables. In this paper we take the initial steps towards enhancing sample efficiency for indirect experiments by adaptively designing a data collection policy. Our main contribution is a practical computational procedure that utilizes influence functions to search for an optimal data collection policy.
  arXiv  Detail & Related papers  (2023-12-05T02:38:04Z)
• Choosing a Proxy Metric from Past Experiments [54.338884612982405]
  In many randomized experiments, the treatment effect of the long-term metric is often difficult or infeasible to measure. A common alternative is to measure several short-term proxy metrics in the hope they closely track the long-term metric. We introduce a new statistical framework to both define and construct an optimal proxy metric for use in a homogeneous population of randomized experiments.
  arXiv  Detail & Related papers  (2023-09-14T17:43:02Z)
• Scalable method for Bayesian experimental design without integrating over posterior distribution [0.0]
  We address the computational efficiency in solving the A-optimal Bayesian design of experiments problems. A-optimality is a widely used and easy-to-interpret criterion for Bayesian experimental design. This study presents a novel likelihood-free approach to the A-optimal experimental design.
  arXiv  Detail & Related papers  (2023-06-30T12:40:43Z)
• Design Amortization for Bayesian Optimal Experimental Design [70.13948372218849]
  We build off of successful variational approaches, which optimize a parameterized variational model with respect to bounds on the expected information gain (EIG) We present a novel neural architecture that allows experimenters to optimize a single variational model that can estimate the EIG for potentially infinitely many designs.
  arXiv  Detail & Related papers  (2022-10-07T02:12:34Z)
• Heavy-tailed Streaming Statistical Estimation [58.70341336199497]
  We consider the task of heavy-tailed statistical estimation given streaming $p$ samples. We design a clipped gradient descent and provide an improved analysis under a more nuanced condition on the noise of gradients.
  arXiv  Detail & Related papers  (2021-08-25T21:30:27Z)
• Increasing the efficiency of randomized trial estimates via linear adjustment for a prognostic score [59.75318183140857]
  Estimating causal effects from randomized experiments is central to clinical research. Most methods for historical borrowing achieve reductions in variance by sacrificing strict type-I error rate control.
  arXiv  Detail & Related papers  (2020-12-17T21:10:10Z)
• Efficient Adaptive Experimental Design for Average Treatment Effect Estimation [18.027128141189355]
  We propose an algorithm for efficient experiments with estimators constructed from dependent samples. To justify our proposed approach, we provide finite and infinite sample analyses.
  arXiv  Detail & Related papers  (2020-02-13T02:04:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.