Linear Contextual Bandits with Interference

• URL: http://arxiv.org/abs/2409.15682v1
• Date: Tue, 24 Sep 2024 02:51:00 GMT
• Title: Linear Contextual Bandits with Interference
• Authors: Yang Xu, Wenbin Lu, Rui Song,
• Abstract summary: We introduce a systematic framework to address interference in Linear CB (LinCB) We propose a series of algorithms that explicitly quantify the interference effect in the reward modeling process. The effectiveness of our approach is demonstrated through simulations and a synthetic data generated based on MovieLens data.
• Score: 14.835167982538053
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Interference, a key concept in causal inference, extends the reward modeling process by accounting for the impact of one unit's actions on the rewards of others. In contextual bandit (CB) settings, where multiple units are present in the same round, potential interference can significantly affect the estimation of expected rewards for different arms, thereby influencing the decision-making process. Although some prior work has explored multi-agent and adversarial bandits in interference-aware settings, the effect of interference in CB, as well as the underlying theory, remains significantly underexplored. In this paper, we introduce a systematic framework to address interference in Linear CB (LinCB), bridging the gap between causal inference and online decision-making. We propose a series of algorithms that explicitly quantify the interference effect in the reward modeling process and provide comprehensive theoretical guarantees, including sublinear regret bounds, finite sample upper bounds, and asymptotic properties. The effectiveness of our approach is demonstrated through simulations and a synthetic data generated based on MovieLens data.

Related papers

• Can We Validate Counterfactual Estimations in the Presence of General Network Interference? [6.092214762701847]
  We introduce a new framework enabling cross-validation for counterfactual estimation. At its core is our distribution-preserving network bootstrap method. We extend recent causal message-passing developments by incorporating heterogeneous unit-level characteristics.
  arXiv  Detail & Related papers  (2025-02-03T06:51:04Z)
• Online Experimental Design With Estimation-Regret Trade-off Under Network Interference [7.080131271060764]
  We introduce a unified interference-aware framework for online experimental design. Compared to existing studies, we extend the definition of arm space by utilizing the statistical concept of exposure mapping. We also propose an algorithmic implementation and discuss its generalization across different learning settings and network topology.
  arXiv  Detail & Related papers  (2024-12-04T21:45:35Z)
• Causal Influence in Federated Edge Inference [34.487472866247586]
  In this paper, we consider a setting where heterogeneous agents with connectivity are performing inference using unlabeled streaming data. In order to overcome the uncertainty, agents cooperate with each other by exchanging their local inferences with and through a fusion center. Various scenarios reflecting different agent participation patterns and fusion center policies are investigated.
  arXiv  Detail & Related papers  (2024-05-02T13:06:50Z)
• Causal Message Passing for Experiments with Unknown and General Network Interference [5.294604210205507]
  We introduce a new framework to accommodate complex and unknown network interference. Our framework, termed causal message-passing, is grounded in high-dimensional approximate message passing methodology. We demonstrate the effectiveness of this approach across five numerical scenarios.
  arXiv  Detail & Related papers  (2023-11-14T17:31:50Z)
• Advancing Counterfactual Inference through Nonlinear Quantile Regression [77.28323341329461]
  We propose a framework for efficient and effective counterfactual inference implemented with neural networks. The proposed approach enhances the capacity to generalize estimated counterfactual outcomes to unseen data. Empirical results conducted on multiple datasets offer compelling support for our theoretical assertions.
  arXiv  Detail & Related papers  (2023-06-09T08:30:51Z)
• Neighborhood Adaptive Estimators for Causal Inference under Network Interference [152.4519491244279]
  We consider the violation of the classical no-interference assumption, meaning that the treatment of one individuals might affect the outcomes of another. To make interference tractable, we consider a known network that describes how interference may travel. We study estimators for the average direct treatment effect on the treated in such a setting.
  arXiv  Detail & Related papers  (2022-12-07T14:53:47Z)
• Inference and Denoise: Causal Inference-based Neural Speech Enhancement [83.4641575757706]
  This study addresses the speech enhancement (SE) task within the causal inference paradigm by modeling the noise presence as an intervention. The proposed causal inference-based speech enhancement (CISE) separates clean and noisy frames in an intervened noisy speech using a noise detector and assigns both sets of frames to two mask-based enhancement modules (EMs) to perform noise-conditional SE.
  arXiv  Detail & Related papers  (2022-11-02T15:03:50Z)
• Algorithmic Recourse in Partially and Fully Confounded Settings Through Bounding Counterfactual Effects [0.6299766708197883]
  Algorithmic recourse aims to provide actionable recommendations to individuals to obtain a more favourable outcome from an automated decision-making system. Existing methods compute the effect of recourse actions using a causal model learnt from data under the assumption of no hidden confounding and modelling assumptions such as additive noise. We propose an alternative approach for discrete random variables which relaxes these assumptions and allows for unobserved confounding and arbitrary structural equations.
  arXiv  Detail & Related papers  (2021-06-22T15:07:49Z)
• Causal Inference Q-Network: Toward Resilient Reinforcement Learning [57.96312207429202]
  We consider a resilient DRL framework with observational interferences. Under this framework, we propose a causal inference based DRL algorithm called causal inference Q-network (CIQ) Our experimental results show that the proposed CIQ method could achieve higher performance and more resilience against observational interferences.
  arXiv  Detail & Related papers  (2021-02-18T23:50:20Z)
• Loss Bounds for Approximate Influence-Based Abstraction [81.13024471616417]
  Influence-based abstraction aims to gain leverage by modeling local subproblems together with the 'influence' that the rest of the system exerts on them. This paper investigates the performance of such approaches from a theoretical perspective. We show that neural networks trained with cross entropy are well suited to learn approximate influence representations.
  arXiv  Detail & Related papers  (2020-11-03T15:33:10Z)
• Estimating the Effects of Continuous-valued Interventions using Generative Adversarial Networks [103.14809802212535]
  We build on the generative adversarial networks (GANs) framework to address the problem of estimating the effect of continuous-valued interventions. Our model, SCIGAN, is flexible and capable of simultaneously estimating counterfactual outcomes for several different continuous interventions. To address the challenges presented by shifting to continuous interventions, we propose a novel architecture for our discriminator.
  arXiv  Detail & Related papers  (2020-02-27T18:46:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.