Task-specific experimental design for treatment effect estimation

• URL: http://arxiv.org/abs/2306.05484v1
• Date: Thu, 8 Jun 2023 18:10:37 GMT
• Title: Task-specific experimental design for treatment effect estimation
• Authors: Bethany Connolly, Kim Moore, Tobias Schwedes, Alexander Adam, Gary Willis, Ilya Feige, Christopher Frye
• Abstract summary: Large randomised trials (RCTs) are the standard for causal inference. Recent work has proposed more sample-efficient alternatives to RCTs, but these are not adaptable to the downstream application for which the causal effect is sought. We develop a task-specific approach to experimental design and derive sampling strategies customised to particular downstream applications.
• Score: 59.879567967089145
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Understanding causality should be a core requirement of any attempt to build real impact through AI. Due to the inherent unobservability of counterfactuals, large randomised trials (RCTs) are the standard for causal inference. But large experiments are generically expensive, and randomisation carries its own costs, e.g. when suboptimal decisions are trialed. Recent work has proposed more sample-efficient alternatives to RCTs, but these are not adaptable to the downstream application for which the causal effect is sought. In this work, we develop a task-specific approach to experimental design and derive sampling strategies customised to particular downstream applications. Across a range of important tasks, real-world datasets, and sample sizes, our method outperforms other benchmarks, e.g. requiring an order-of-magnitude less data to match RCT performance on targeted marketing tasks.

Related papers

• Adaptive Experimentation When You Can't Experiment [55.86593195947978]
  This paper introduces the emphconfounded pure exploration transductive linear bandit (textttCPET-LB) problem. Online services can employ a properly randomized encouragement that incentivizes users toward a specific treatment.
  arXiv  Detail & Related papers  (2024-06-15T20:54:48Z)
• IncomeSCM: From tabular data set to time-series simulator and causal estimation benchmark [8.960138649105156]
  We propose a strategy for turning observational data into sequential structural causal models. We apply these ideas to the well-known Adult income data set to construct the IncomeSCM simulator. The tasks present a suitable challenge, with effect estimates varying greatly in quality between methods.
  arXiv  Detail & Related papers  (2024-05-25T05:40:16Z)
• An Experimental Design Framework for Label-Efficient Supervised Finetuning of Large Language Models [55.01592097059969]
  Supervised finetuning on instruction datasets has played a crucial role in achieving the remarkable zero-shot generalization capabilities. Active learning is effective in identifying useful subsets of samples to annotate from an unlabeled pool. We propose using experimental design to circumvent the computational bottlenecks of active learning.
  arXiv  Detail & Related papers  (2024-01-12T16:56:54Z)
• Adaptive Instrument Design for Indirect Experiments [48.815194906471405]
  Unlike RCTs, indirect experiments estimate treatment effects by leveragingconditional instrumental variables. In this paper we take the initial steps towards enhancing sample efficiency for indirect experiments by adaptively designing a data collection policy. Our main contribution is a practical computational procedure that utilizes influence functions to search for an optimal data collection policy.
  arXiv  Detail & Related papers  (2023-12-05T02:38:04Z)
• RCT Rejection Sampling for Causal Estimation Evaluation [25.845034753006367]
  Confounding is a significant obstacle to unbiased estimation of causal effects from observational data. We build on a promising empirical evaluation strategy that simplifies evaluation design and uses real data. We show our algorithm indeed results in low bias when oracle estimators are evaluated on confounded samples.
  arXiv  Detail & Related papers  (2023-07-27T20:11:07Z)
• Delving into Identify-Emphasize Paradigm for Combating Unknown Bias [52.76758938921129]
  We propose an effective bias-conflicting scoring method (ECS) to boost the identification accuracy. We also propose gradient alignment (GA) to balance the contributions of the mined bias-aligned and bias-conflicting samples. Experiments are conducted on multiple datasets in various settings, demonstrating that the proposed solution can mitigate the impact of unknown biases.
  arXiv  Detail & Related papers  (2023-02-22T14:50:24Z)
• Active Learning for Optimal Intervention Design in Causal Models [11.294389953686945]
  We develop a causal active learning strategy to identify interventions that are optimal, as measured by the discrepancy between the post-interventional mean of the distribution and a desired target mean. We apply our approach to both synthetic data and single-cell transcriptomic data from Perturb-CITE-seq experiments to identify optimal perturbations that induce a specific cell state transition.
  arXiv  Detail & Related papers  (2022-09-10T20:40:30Z)
• Reinforcement Learning based Sequential Batch-sampling for Bayesian Optimal Experimental Design [1.6249267147413522]
  Sequential design of experiments (SDOE) is a popular suite of methods, that has yielded promising results in recent years. In this work, we aim to extend the SDOE strategy, to query the experiment or computer code at a batch of inputs. A unique capability of the proposed methodology is its ability to be applied to multiple tasks, for example optimization of a function, once its trained.
  arXiv  Detail & Related papers  (2021-12-21T02:25:23Z)
• Constrained multi-objective optimization of process design parameters in settings with scarce data: an application to adhesive bonding [48.7576911714538]
  Finding the optimal process parameters for an adhesive bonding process is challenging. Traditional evolutionary approaches (such as genetic algorithms) are then ill-suited to solve the problem. In this research, we successfully applied specific machine learning techniques to emulate the objective and constraint functions.
  arXiv  Detail & Related papers  (2021-12-16T10:14:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.