A Two-Part Machine Learning Approach to Characterizing Network Interference in A/B Testing

• URL: http://arxiv.org/abs/2308.09790v2
• Date: Sat, 29 Jun 2024 05:28:23 GMT
• Title: A Two-Part Machine Learning Approach to Characterizing Network Interference in A/B Testing
• Authors: Yuan Yuan, Kristen M. Altenburger,
• Abstract summary: We introduce "causal network motifs" and utilize transparent machine learning models to characterize network interference patterns. Our approach provides a comprehensive and automated solution to address network interference for A/B testing practitioners.
• Score: 4.000213034401085
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The reliability of controlled experiments, commonly referred to as "A/B tests," is often compromised by network interference, where the outcomes of individual units are influenced by interactions with others. Significant challenges in this domain include the lack of accounting for complex social network structures and the difficulty in suitably characterizing network interference. To address these challenges, we propose a machine learning-based method. We introduce "causal network motifs" and utilize transparent machine learning models to characterize network interference patterns underlying an A/B test on networks. Our method's performance has been demonstrated through simulations on both a synthetic experiment and a large-scale test on Instagram. Our experiments show that our approach outperforms conventional methods such as design-based cluster randomization and conventional analysis-based neighborhood exposure mapping. Our approach provides a comprehensive and automated solution to address network interference for A/B testing practitioners. This aids in informing strategic business decisions in areas such as marketing effectiveness and product customization.

Related papers

• Cross-Domain Continual Learning via CLAMP [10.553456651003055]
  CLAMP significantly outperforms established baseline algorithms across all experiments by at least $10%$ margin. An assessor-guided learning process is put forward to navigate the learning process of a base model.
  arXiv  Detail & Related papers  (2024-05-12T02:41:31Z)
• Graph Machine Learning based Doubly Robust Estimator for Network Causal Effects [17.44202934049009]
  We propose a novel methodology that combines graph machine learning approaches with the double machine learning framework. We demonstrate our method is accurate, robust, and scalable via an extensive simulation study.
  arXiv  Detail & Related papers  (2024-03-17T20:23:42Z)
• Effective Intrusion Detection in Heterogeneous Internet-of-Things Networks via Ensemble Knowledge Distillation-based Federated Learning [52.6706505729803]
  We introduce Federated Learning (FL) to collaboratively train a decentralized shared model of Intrusion Detection Systems (IDS) FLEKD enables a more flexible aggregation method than conventional model fusion techniques. Experiment results show that the proposed approach outperforms local training and traditional FL in terms of both speed and performance.
  arXiv  Detail & Related papers  (2024-01-22T14:16:37Z)
• Adversarial Machine Learning In Network Intrusion Detection Domain: A Systematic Review [0.0]
  It has been found that deep learning models are vulnerable to data instances that can mislead the model to make incorrect classification decisions. This survey explores the researches that employ different aspects of adversarial machine learning in the area of network intrusion detection.
  arXiv  Detail & Related papers  (2021-12-06T19:10:23Z)
• Towards Unbiased Visual Emotion Recognition via Causal Intervention [63.74095927462]
  We propose a novel Emotion Recognition Network (IERN) to alleviate the negative effects brought by the dataset bias. A series of designed tests validate the effectiveness of IERN, and experiments on three emotion benchmarks demonstrate that IERN outperforms other state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-07-26T10:40:59Z)
• Learning Interpretable Models for Coupled Networks Under Domain Constraints [8.308385006727702]
  We investigate the idea of coupled networks by focusing on interactions between structural edges and functional edges of brain networks. We propose a novel formulation to place hard network constraints on the noise term while estimating interactions. We validate our method on multishell diffusion and task-evoked fMRI datasets from the Human Connectome Project.
  arXiv  Detail & Related papers  (2021-04-19T06:23:31Z)
• Explainable Adversarial Attacks in Deep Neural Networks Using Activation Profiles [69.9674326582747]
  This paper presents a visual framework to investigate neural network models subjected to adversarial examples. We show how observing these elements can quickly pinpoint exploited areas in a model.
  arXiv  Detail & Related papers  (2021-03-18T13:04:21Z)
• Anomaly Detection on Attributed Networks via Contrastive Self-Supervised Learning [50.24174211654775]
  We present a novel contrastive self-supervised learning framework for anomaly detection on attributed networks. Our framework fully exploits the local information from network data by sampling a novel type of contrastive instance pair. A graph neural network-based contrastive learning model is proposed to learn informative embedding from high-dimensional attributes and local structure.
  arXiv  Detail & Related papers  (2021-02-27T03:17:20Z)
• Increasing the Confidence of Deep Neural Networks by Coverage Analysis [71.57324258813674]
  This paper presents a lightweight monitoring architecture based on coverage paradigms to enhance the model against different unsafe inputs. Experimental results show that the proposed approach is effective in detecting both powerful adversarial examples and out-of-distribution inputs.
  arXiv  Detail & Related papers  (2021-01-28T16:38:26Z)
• DNS Covert Channel Detection via Behavioral Analysis: a Machine Learning Approach [0.09176056742068815]
  We propose an effective covert channel detection method based on the analysis of DNS network data passively extracted from a network monitoring system. The proposed solution has been evaluated over a 15-day-long experimental session with the injection of traffic that covers the most relevant exfiltration and tunneling attacks.
  arXiv  Detail & Related papers  (2020-10-04T13:28:28Z)
• Vulnerability Under Adversarial Machine Learning: Bias or Variance? [77.30759061082085]
  We investigate the effect of adversarial machine learning on the bias and variance of a trained deep neural network. Our analysis sheds light on why the deep neural networks have poor performance under adversarial perturbation. We introduce a new adversarial machine learning algorithm with lower computational complexity than well-known adversarial machine learning strategies.
  arXiv  Detail & Related papers  (2020-08-01T00:58:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.