Networked Restless Multi-Armed Bandits for Mobile Interventions

• URL: http://arxiv.org/abs/2201.12408v1
• Date: Fri, 28 Jan 2022 20:38:01 GMT
• Title: Networked Restless Multi-Armed Bandits for Mobile Interventions
• Authors: Han-Ching Ou, Christoph Siebenbrunner, Jackson Killian, Meredith B Brooks, David Kempe, Yevgeniy Vorobeychik, Milind Tambe
• Abstract summary: We study restless multi-armed bandits (RMABs) with network effects. In our model, arms are partially recharging and connected through a graph, so that pulling one arm also improves the state of neighboring arms. We show that network effects in RMABs induce strong reward coupling that is not accounted for by existing solution methods.
• Score: 41.74987432512137
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Motivated by a broad class of mobile intervention problems, we propose and study restless multi-armed bandits (RMABs) with network effects. In our model, arms are partially recharging and connected through a graph, so that pulling one arm also improves the state of neighboring arms, significantly extending the previously studied setting of fully recharging bandits with no network effects. In mobile interventions, network effects may arise due to regular population movements (such as commuting between home and work). We show that network effects in RMABs induce strong reward coupling that is not accounted for by existing solution methods. We propose a new solution approach for networked RMABs, exploiting concavity properties which arise under natural assumptions on the structure of intervention effects. We provide sufficient conditions for optimality of our approach in idealized settings and demonstrate that it empirically outperforms state-of-the art baselines in three mobile intervention domains using real-world graphs.

Related papers

• Why Steering Works: Toward a Unified View of Language Model Parameter Dynamics [81.80010043113445]
  Local weight fine-tuning, LoRA-based adaptation, and activation-based interventions are studied in isolation.<n>We present a unified view that frames these interventions as dynamic weight updates induced by a control signal.<n>Across methods, we observe a consistent trade-off between preference and utility: stronger control increases preference while predictably reducing utility.
  arXiv  Detail & Related papers  (2026-02-02T17:04:36Z)
• Networked Restless Multi-Arm Bandits with Reinforcement Learning [4.0539039756740785]
  This paper introduces Networked RMAB, a novel framework that integrates the RMAB model with the independent cascade model.<n>We present its computational challenge due to exponentially large action and state spaces.<n>We experimentally verify these results by developing an efficient Q-learning algorithm tailored to the networked setting.
  arXiv  Detail & Related papers  (2025-12-06T03:53:25Z)
• Constrained Adversarial Perturbation [16.05659740749269]
  Universal Adversarial Perturbations (UAPs) have emerged as a powerful tool for both stress testing model robustness and scalable adversarial training.<n>We propose Constrained Adversarial Perturbation (CAP), an efficient algorithm that solves this problem using a gradient based alternating optimization strategy.
  arXiv  Detail & Related papers  (2025-10-17T14:44:20Z)
• Learning Robust Intervention Representations with Delta Embeddings [5.124256074746721]
  Causal representation learning has attracted significant research interest during the past few years.<n>We show that an effective strategy for improving out of distribution robustness is to focus on the representation of interventions in the latent space.<n>We propose a framework that is capable of learning causal representations from image pairs, without any additional supervision.
  arXiv  Detail & Related papers  (2025-08-06T14:39:34Z)
• Efficient and Trustworthy Block Propagation for Blockchain-enabled Mobile Embodied AI Networks: A Graph Resfusion Approach [60.80257080226662]
  We propose a graph Resfusion model-based trustworthy block propagation optimization framework for consortium blockchain-enabled MEANETs. Specifically, we propose an innovative trust calculation mechanism based on the trust cloud model. By leveraging the strengths of graph neural networks and diffusion models, we develop a graph Resfusion model to effectively and adaptively generate the optimal block propagation trajectory.
  arXiv  Detail & Related papers  (2025-01-26T07:47:05Z)
• Influence Maximization via Graph Neural Bandits [54.45552721334886]
  We set the IM problem in a multi-round diffusion campaign, aiming to maximize the number of distinct users that are influenced. We propose the framework IM-GNB (Influence Maximization with Graph Neural Bandits), where we provide an estimate of the users' probabilities of being influenced.
  arXiv  Detail & Related papers  (2024-06-18T17:54:33Z)
• Distributed Autonomous Swarm Formation for Dynamic Network Bridging [40.27919181139919]
  We formulate the problem of dynamic network bridging in a novel Decentralized Partially Observable Markov Decision Process (Dec-POMDP) We propose a Multi-Agent Reinforcement Learning (MARL) approach for the problem based on Graph Convolutional Reinforcement Learning (DGN) The proposed method is evaluated in a simulated environment and compared to a centralized baseline showing promising results.
  arXiv  Detail & Related papers  (2024-04-02T01:45:03Z)
• Exploiting Regional Information Transformer for Single Image Deraining [40.96287901893822]
  Region Transformer Block (RTB) integrates a Region Masked Attention (RMA) mechanism and a Mixed Gate Forward Block (MGFB) Our model reaches state-of-the-art performance, significantly improving the image deraining quality.
  arXiv  Detail & Related papers  (2024-02-25T09:09:30Z)
• Towards a Pretrained Model for Restless Bandits via Multi-arm Generalization [32.90636136408938]
  Restless multi-arm bandits (RMABs) are resource allocation problems with broad application in areas such as healthcare, online advertising, and anti-poaching. We develop a neural network-based pre-trained model (PreFeRMAB) that has general zero-shot ability on a wide range of previously unseen RMABs.
  arXiv  Detail & Related papers  (2023-10-23T03:16:32Z)
• Leveraging Low-Rank and Sparse Recurrent Connectivity for Robust Closed-Loop Control [63.310780486820796]
  We show how a parameterization of recurrent connectivity influences robustness in closed-loop settings. We find that closed-form continuous-time neural networks (CfCs) with fewer parameters can outperform their full-rank, fully-connected counterparts.
  arXiv  Detail & Related papers  (2023-10-05T21:44:18Z)
• Model-based Causal Bayesian Optimization [74.78486244786083]
  We introduce the first algorithm for Causal Bayesian Optimization with Multiplicative Weights (CBO-MW) We derive regret bounds for CBO-MW that naturally depend on graph-related quantities. Our experiments include a realistic demonstration of how CBO-MW can be used to learn users' demand patterns in a shared mobility system.
  arXiv  Detail & Related papers  (2023-07-31T13:02:36Z)
• Networked Restless Bandits with Positive Externalities [34.792869761921565]
  We introduce networked restless bandits, a novel multi-armed bandit setting in which arms are both restless and embedded within a directed graph. We then present Greta, a graph-aware, Whittle index-based algorithm that can be used to efficiently construct a constrained reward-maximizing action vector at each timestep.
  arXiv  Detail & Related papers  (2022-12-09T23:37:14Z)
• Low-Latency Federated Learning over Wireless Channels with Differential Privacy [142.5983499872664]
  In federated learning (FL), model training is distributed over clients and local models are aggregated by a central server. In this paper, we aim to minimize FL training delay over wireless channels, constrained by overall training performance as well as each client's differential privacy (DP) requirement.
  arXiv  Detail & Related papers  (2021-06-20T13:51:18Z)
• On Topology Optimization and Routing in Integrated Access and Backhaul Networks: A Genetic Algorithm-based Approach [70.85399600288737]
  We study the problem of topology optimization and routing in IAB networks. We develop efficient genetic algorithm-based schemes for both IAB node placement and non-IAB backhaul link distribution. We discuss the main challenges for enabling mesh-based IAB networks.
  arXiv  Detail & Related papers  (2021-02-14T21:52:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.