Bayesian Ego-graph inference for Networked Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2509.16606v2
• Date: Fri, 17 Oct 2025 03:31:04 GMT
• Title: Bayesian Ego-graph inference for Networked Multi-Agent Reinforcement Learning
• Authors: Wei Duan, Jie Lu, Junyu Xuan,
• Abstract summary: We propose a graph-based policy for Networked-MARL, where each agent conditions its decision on a sampled subgraph over its local physical neighborhood.<n>We introduce BayesG, a decentralized actor-framework that learns sparse, context-aware interaction structures via Bayesian variational inference.<n>BayesG outperforms strong MARL baselines on large-scale traffic control tasks with up to 167 agents.
• Score: 16.190458233440864
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In networked multi-agent reinforcement learning (Networked-MARL), decentralized agents must act under local observability and constrained communication over fixed physical graphs. Existing methods often assume static neighborhoods, limiting adaptability to dynamic or heterogeneous environments. While centralized frameworks can learn dynamic graphs, their reliance on global state access and centralized infrastructure is impractical in real-world decentralized systems. We propose a stochastic graph-based policy for Networked-MARL, where each agent conditions its decision on a sampled subgraph over its local physical neighborhood. Building on this formulation, we introduce BayesG, a decentralized actor-framework that learns sparse, context-aware interaction structures via Bayesian variational inference. Each agent operates over an ego-graph and samples a latent communication mask to guide message passing and policy computation. The variational distribution is trained end-to-end alongside the policy using an evidence lower bound (ELBO) objective, enabling agents to jointly learn both interaction topology and decision-making strategies. BayesG outperforms strong MARL baselines on large-scale traffic control tasks with up to 167 agents, demonstrating superior scalability, efficiency, and performance.

Related papers

• Multi-Agent Deep Reinforcement Learning Under Constrained Communications [2.7126292487109005]
  We present a distributed multi-agent reinforcement learning (MARL) framework that removes the need for centralized critics or global information.<n>We develop a novel Graph Attention Network (D-GAT) that performs global state inference through multi-hop communication.<n>We also develop the distributed graph-attention MAPPO (DG-MAPPO) -- a distributed MARL framework where agents optimize local policies and value functions.
  arXiv  Detail & Related papers  (2026-01-22T21:07:18Z)
• Power Grid Control with Graph-Based Distributed Reinforcement Learning [60.49805771047161]
  This work advances a graph-based distributed reinforcement learning framework for real-time, scalable grid management.<n>A Graph Neural Network (GNN) is employed to encode the network's topological information within the single low-level agent's observation.<n>Experiments on the Grid2Op simulation environment show the effectiveness of the approach.
  arXiv  Detail & Related papers  (2025-09-02T22:17:25Z)
• Q-MARL: A quantum-inspired algorithm using neural message passing for large-scale multi-agent reinforcement learning [27.014941513702183]
  Q-MARL is a completely decentralised learning architecture.<n>It supports very large-scale multi-agent reinforcement learning scenarios.<n>A detailed theoretical analysis proves improvement and convergence.
  arXiv  Detail & Related papers  (2025-03-10T14:43:10Z)
• Decentralized Learning Strategies for Estimation Error Minimization with Graph Neural Networks [94.2860766709971]
  We address the challenge of sampling and remote estimation for autoregressive Markovian processes in a wireless network with statistically-identical agents.<n>Our goal is to minimize time-average estimation error and/or age of information with decentralized scalable sampling and transmission policies.
  arXiv  Detail & Related papers  (2024-04-04T06:24:11Z)
• Federated Multi-Agent Mapping for Planetary Exploration [0.4143603294943439]
  We propose a federated multi-agent mapping approach that jointly trains a global map model across agents without transmitting raw data.<n>Our method leverages implicit neural mapping to generate parsimonious, adaptable representations, reducing data transmission by up to 93.8% compared to raw maps.<n>We demonstrate the efficacy of our approach on Martian terrains and glacier datasets, achieving downstream path planning F1 scores as high as 0.95 while outperforming on map reconstruction losses.
  arXiv  Detail & Related papers  (2024-04-02T20:32:32Z)
• Mean-Field Multi-Agent Reinforcement Learning: A Decentralized Network Approach [6.802025156985356]
  This paper proposes a framework called localized training and decentralized execution to study MARL with network of states. The key idea is to utilize the homogeneity of agents and regroup them according to their states, thus the formulation of a networked Markov decision process.
  arXiv  Detail & Related papers  (2021-08-05T16:52:36Z)
• Learning Connectivity for Data Distribution in Robot Teams [96.39864514115136]
  We propose a task-agnostic, decentralized, low-latency method for data distribution in ad-hoc networks using Graph Neural Networks (GNN) Our approach enables multi-agent algorithms based on global state information to function by ensuring it is available at each robot. We train the distributed GNN communication policies via reinforcement learning using the average Age of Information as the reward function and show that it improves training stability compared to task-specific reward functions.
  arXiv  Detail & Related papers  (2021-03-08T21:48:55Z)
• BayGo: Joint Bayesian Learning and Information-Aware Graph Optimization [48.30183416069897]
  BayGo is a novel fully decentralized joint Bayesian learning and graph optimization framework. We show that our framework achieves faster convergence and higher accuracy compared to fully-connected and star topology graphs.
  arXiv  Detail & Related papers  (2020-11-09T11:16:55Z)
• Dif-MAML: Decentralized Multi-Agent Meta-Learning [54.39661018886268]
  We propose a cooperative multi-agent meta-learning algorithm, referred to as MAML or Dif-MAML. We show that the proposed strategy allows a collection of agents to attain agreement at a linear rate and to converge to a stationary point of the aggregate MAML. Simulation results illustrate the theoretical findings and the superior performance relative to the traditional non-cooperative setting.
  arXiv  Detail & Related papers  (2020-10-06T16:51:09Z)
• F2A2: Flexible Fully-decentralized Approximate Actor-critic for Cooperative Multi-agent Reinforcement Learning [110.35516334788687]
  Decentralized multi-agent reinforcement learning algorithms are sometimes unpractical in complicated applications. We propose a flexible fully decentralized actor-critic MARL framework, which can handle large-scale general cooperative multi-agent setting. Our framework can achieve scalability and stability for large-scale environment and reduce information transmission.
  arXiv  Detail & Related papers  (2020-04-17T14:56:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.