Minimizing Interference and Selection Bias in Network Experiment Design

• URL: http://arxiv.org/abs/2004.07225v1
• Date: Wed, 15 Apr 2020 17:34:13 GMT
• Title: Minimizing Interference and Selection Bias in Network Experiment Design
• Authors: Zahra Fatemi, Elena Zheleva
• Abstract summary: We propose a principled framework for network experiment design which jointly minimizes interference and selection bias. Our experiments on a number of real-world datasets show that our proposed framework leads to significantly lower error in causal effect estimation.
• Score: 14.696233190562939
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Current approaches to A/B testing in networks focus on limiting interference, the concern that treatment effects can "spill over" from treatment nodes to control nodes and lead to biased causal effect estimation. Prominent methods for network experiment design rely on two-stage randomization, in which sparsely-connected clusters are identified and cluster randomization dictates the node assignment to treatment and control. Here, we show that cluster randomization does not ensure sufficient node randomization and it can lead to selection bias in which treatment and control nodes represent different populations of users. To address this problem, we propose a principled framework for network experiment design which jointly minimizes interference and selection bias. We introduce the concepts of edge spillover probability and cluster matching and demonstrate their importance for designing network A/B testing. Our experiments on a number of real-world datasets show that our proposed framework leads to significantly lower error in causal effect estimation than existing solutions.

Related papers

• Differences-in-Neighbors for Network Interference in Experiments [5.079602839359523]
  We propose a new estimator, dubbed Differences-in-Neighbors (DN), designed explicitly to mitigate network interference. Compared to DM estimators, DN bias second order in the magnitude of the interference effect, while its variance is exponentially smaller than that of HT estimators. Empirical evaluations on a large-scale social network and a city-level ride-sharing simulator demonstrate DN's superior performance.
  arXiv  Detail & Related papers  (2025-03-04T04:40:12Z)
• Can We Validate Counterfactual Estimations in the Presence of General Network Interference? [6.092214762701847]
  We introduce a new framework enabling cross-validation for counterfactual estimation. At its core is our distribution-preserving network bootstrap method. We extend recent causal message-passing developments by incorporating heterogeneous unit-level characteristics.
  arXiv  Detail & Related papers  (2025-02-03T06:51:04Z)
• Online Experimental Design With Estimation-Regret Trade-off Under Network Interference [7.080131271060764]
  We introduce a unified interference-aware framework for online experimental design. Compared to existing studies, we extend the definition of arm space by utilizing the statistical concept of exposure mapping. We also propose an algorithmic implementation and discuss its generalization across different learning settings and network topology.
  arXiv  Detail & Related papers  (2024-12-04T21:45:35Z)
• Cascade-based Randomization for Inferring Causal Effects under Diffusion Interference [15.7485894481935]
  Cluster-based randomization approaches perform poorly when interference propagates in cascades. We propose a cascade-based network experiment design that initiates treatment assignment from the cascade seed node and propagates the assignment to their multi-hop neighbors.
  arXiv  Detail & Related papers  (2024-05-20T19:24:10Z)
• Large-Scale Sequential Learning for Recommender and Engineering Systems [91.3755431537592]
  In this thesis, we focus on the design of an automatic algorithms that provide personalized ranking by adapting to the current conditions. For the former, we propose novel algorithm called SAROS that take into account both kinds of feedback for learning over the sequence of interactions. The proposed idea of taking into account the neighbour lines shows statistically significant results in comparison with the initial approach for faults detection in power grid.
  arXiv  Detail & Related papers  (2022-05-13T21:09:41Z)
• Layer Adaptive Node Selection in Bayesian Neural Networks: Statistical Guarantees and Implementation Details [0.5156484100374059]
  Sparse deep neural networks have proven to be efficient for predictive model building in large-scale studies. We propose a Bayesian sparse solution using spike-and-slab Gaussian priors to allow for node selection during training. We establish the fundamental result of variational posterior consistency together with the characterization of prior parameters.
  arXiv  Detail & Related papers  (2021-08-25T00:48:07Z)
• Deep Random Projection Outlyingness for Unsupervised Anomaly Detection [1.2249546377051437]
  The original random projection outlyingness measure is modified and associated with a neural network to obtain an unsupervised anomaly detection method. The performance of the proposed neural network approach is comparable to a state-of-the-art anomaly detection method.
  arXiv  Detail & Related papers  (2021-06-08T14:13:43Z)
• Adversarial Examples Detection with Bayesian Neural Network [57.185482121807716]
  We propose a new framework to detect adversarial examples motivated by the observations that random components can improve the smoothness of predictors. We propose a novel Bayesian adversarial example detector, short for BATer, to improve the performance of adversarial example detection.
  arXiv  Detail & Related papers  (2021-05-18T15:51:24Z)
• Deconfounding Scores: Feature Representations for Causal Effect Estimation with Weak Overlap [140.98628848491146]
  We introduce deconfounding scores, which induce better overlap without biasing the target of estimation. We show that deconfounding scores satisfy a zero-covariance condition that is identifiable in observed data. In particular, we show that this technique could be an attractive alternative to standard regularizations.
  arXiv  Detail & Related papers  (2021-04-12T18:50:11Z)
• Sampling-free Variational Inference for Neural Networks with Multiplicative Activation Noise [51.080620762639434]
  We propose a more efficient parameterization of the posterior approximation for sampling-free variational inference. Our approach yields competitive results for standard regression problems and scales well to large-scale image classification tasks.
  arXiv  Detail & Related papers  (2021-03-15T16:16:18Z)
• Bayesian Nested Neural Networks for Uncertainty Calibration and Adaptive Compression [40.35734017517066]
  Nested networks or slimmable networks are neural networks whose architectures can be adjusted instantly during testing time. Recent studies have focused on a "nested dropout" layer, which is able to order the nodes of a layer by importance during training.
  arXiv  Detail & Related papers  (2021-01-27T12:34:58Z)
• Ramifications of Approximate Posterior Inference for Bayesian Deep Learning in Adversarial and Out-of-Distribution Settings [7.476901945542385]
  We show that Bayesian deep learning models on certain occasions marginally outperform conventional neural networks. Preliminary investigations indicate the potential inherent role of bias due to choices of initialisation, architecture or activation functions.
  arXiv  Detail & Related papers  (2020-09-03T16:58:15Z)
• Spatially Adaptive Inference with Stochastic Feature Sampling and Interpolation [72.40827239394565]
  We propose to compute features only at sparsely sampled locations. We then densely reconstruct the feature map with an efficient procedure. The presented network is experimentally shown to save substantial computation while maintaining accuracy over a variety of computer vision tasks.
  arXiv  Detail & Related papers  (2020-03-19T15:36:31Z)
• Almost-Matching-Exactly for Treatment Effect Estimation under Network Interference [73.23326654892963]
  We propose a matching method that recovers direct treatment effects from randomized experiments where units are connected in an observed network. Our method matches units almost exactly on counts of unique subgraphs within their neighborhood graphs.
  arXiv  Detail & Related papers  (2020-03-02T15:21:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.