Confounding is a Pervasive Problem in Real World Recommender Systems

• URL: http://arxiv.org/abs/2508.10479v2
• Date: Mon, 08 Sep 2025 09:26:57 GMT
• Title: Confounding is a Pervasive Problem in Real World Recommender Systems
• Authors: Alexander Merkov, David Rohde, Alexandre Gilotte, Benjamin Heymann,
• Abstract summary: Unobserved confounding undermines observational studies in fields like economics, medicine, ecology or epidemiology.<n>This paper will show that numerous common practices such as feature engineering, A/B testing and modularization can in fact introduce confounding into recommendation systems.
• Score: 78.18849951612815
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Unobserved confounding arises when an unmeasured feature influences both the treatment and the outcome, leading to biased causal effect estimates. This issue undermines observational studies in fields like economics, medicine, ecology or epidemiology. Recommender systems leveraging fully observed data seem not to be vulnerable to this problem. However many standard practices in recommender systems result in observed features being ignored, resulting in effectively the same problem. This paper will show that numerous common practices such as feature engineering, A/B testing and modularization can in fact introduce confounding into recommendation systems and hamper their performance. Several illustrations of the phenomena are provided, supported by simulation studies with practical suggestions about how practitioners may reduce or avoid the affects of confounding in real systems.

Related papers

• Does Unsupervised Domain Adaptation Improve the Robustness of Amortized Bayesian Inference? A Systematic Evaluation [3.4109073456116477]
  Recent robust approaches employ unsupervised domain adaptation (UDA) to match the embedding spaces of simulated and observed data.<n>We demonstrate that aligning summary spaces between domains effectively mitigates the impact of unmodeled phenomena or noise.<n>Our results underscore the need for careful consideration of misspecification types when using UDA to increase the robustness of ABI.
  arXiv  Detail & Related papers  (2025-02-07T14:13:51Z)
• Higher-Order Causal Message Passing for Experimentation with Complex Interference [6.092214762701847]
  We introduce a new class of estimators based on causal message-passing, specifically designed for settings with pervasive, unknown interference.<n>Our estimator draws on information from the sample mean and variance of unit outcomes and treatments over time, enabling efficient use of observed data.
  arXiv  Detail & Related papers  (2024-11-01T18:00:51Z)
• The Blessings of Multiple Treatments and Outcomes in Treatment Effect Estimation [53.81860494566915]
  Existing studies leveraged proxy variables or multiple treatments to adjust for confounding bias. In many real-world scenarios, there is greater interest in studying the effects on multiple outcomes. We show that parallel studies of multiple outcomes involved in this setting can assist each other in causal identification.
  arXiv  Detail & Related papers  (2023-09-29T14:33:48Z)
• Offline Recommender System Evaluation under Unobserved Confounding [5.4208903577329375]
  Off-Policy Estimation methods allow us to learn and evaluate decision-making policies from logged data. An important assumption that makes this work is the absence of unobserved confounders. This work aims to highlight the problems that arise when performing off-policy estimation in the presence of unobserved confounders.
  arXiv  Detail & Related papers  (2023-09-08T09:11:26Z)
• Benchmarking Bayesian Causal Discovery Methods for Downstream Treatment Effect Estimation [137.3520153445413]
  A notable gap exists in the evaluation of causal discovery methods, where insufficient emphasis is placed on downstream inference. We evaluate seven established baseline causal discovery methods including a newly proposed method based on GFlowNets. The results of our study demonstrate that some of the algorithms studied are able to effectively capture a wide range of useful and diverse ATE modes.
  arXiv  Detail & Related papers  (2023-07-11T02:58:10Z)
• A Double Machine Learning Approach to Combining Experimental and Observational Data [59.29868677652324]
  We propose a double machine learning approach to combine experimental and observational studies. Our framework tests for violations of external validity and ignorability under milder assumptions.
  arXiv  Detail & Related papers  (2023-07-04T02:53:11Z)
• CausalBench: A Large-scale Benchmark for Network Inference from Single-cell Perturbation Data [61.088705993848606]
  We introduce CausalBench, a benchmark suite for evaluating causal inference methods on real-world interventional data. CaulBench incorporates biologically-motivated performance metrics, including new distribution-based interventional metrics.
  arXiv  Detail & Related papers  (2022-10-31T13:04:07Z)
• Recency Dropout for Recurrent Recommender Systems [23.210278548403185]
  We introduce the recency dropout technique, a simple yet effective data augmentation technique to alleviate the recency bias in recommender systems. We demonstrate the effectiveness of recency dropout in various experimental settings including a simulation study, offline experiments, as well as live experiments on a large-scale industrial recommendation platform.
  arXiv  Detail & Related papers  (2022-01-26T15:50:20Z)
• SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data [83.50281440043241]
  We study the problem of inferring heterogeneous treatment effects from time-to-event data. We propose a novel deep learning method for treatment-specific hazard estimation based on balancing representations.
  arXiv  Detail & Related papers  (2021-10-26T20:13:17Z)
• Estimating Individual Treatment Effects using Non-Parametric Regression Models: a Review [0.0]
  We introduce the setup and the issues related to conducting causal inference with observational or non-fully randomized data. We develop a unifying taxonomy of the existing state-of-the-art frameworks that allow for individual treatment effects estimation. We conclude by demonstrating the use of some of the methods on an empirical analysis of the school meal program data.
  arXiv  Detail & Related papers  (2020-09-14T14:26:55Z)
• Generalization Bounds and Representation Learning for Estimation of Potential Outcomes and Causal Effects [61.03579766573421]
  We study estimation of individual-level causal effects, such as a single patient's response to alternative medication. We devise representation learning algorithms that minimize our bound, by regularizing the representation's induced treatment group distance. We extend these algorithms to simultaneously learn a weighted representation to further reduce treatment group distances.
  arXiv  Detail & Related papers  (2020-01-21T10:16:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.