Factored Value Functions for Graph-Based Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2601.11401v1
• Date: Fri, 16 Jan 2026 16:11:50 GMT
• Title: Factored Value Functions for Graph-Based Multi-Agent Reinforcement Learning
• Abstract summary: Credit assignment is a core challenge in multi-agent reinforcement learning (MARL)<n>We introduce the Diffusion Value Function (DVF), a factored value function for GMDPs that assigns to each agent a value component by diffusing rewards over the influence graph with temporal discounting and spatial attenuation.<n>We show that DVF is well-defined, admits a Bellman fixed point, and decomposes the global discounted value via an averaging property.<n>We propose Diffusion A2C (DA2C) and a sparse message-passing actor, Learned DropEdge GNN (LD-GNN), for learning decentralised algorithms under communication costs.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Credit assignment is a core challenge in multi-agent reinforcement learning (MARL), especially in large-scale systems with structured, local interactions. Graph-based Markov decision processes (GMDPs) capture such settings via an influence graph, but standard critics are poorly aligned with this structure: global value functions provide weak per-agent learning signals, while existing local constructions can be difficult to estimate and ill-behaved in infinite-horizon settings. We introduce the Diffusion Value Function (DVF), a factored value function for GMDPs that assigns to each agent a value component by diffusing rewards over the influence graph with temporal discounting and spatial attenuation. We show that DVF is well-defined, admits a Bellman fixed point, and decomposes the global discounted value via an averaging property. DVF can be used as a drop-in critic in standard RL algorithms and estimated scalably with graph neural networks. Building on DVF, we propose Diffusion A2C (DA2C) and a sparse message-passing actor, Learned DropEdge GNN (LD-GNN), for learning decentralised algorithms under communication costs. Across the firefighting benchmark and three distributed computation tasks (vector graph colouring and two transmit power optimisation problems), DA2C consistently outperforms local and global critic baselines, improving average reward by up to 11%.

Related papers

• Group-Graph Policy Optimization for Long-Horizon Agentic Reinforcement Learning [85.02566758103008]
  Group-Graph Policy Optimization (G2PO) is a novel group-based Reinforcement Learning algorithm tailored for multi-turn agentic tasks.<n>G2PO substantially outperforms state-of-the-art prompt-based and RL baselines, achieving remarkable success rate improvements of up to 22.2% over GRPO.
  arXiv  Detail & Related papers  (2026-06-22T08:12:47Z)
• Sketching the Readout of Large Language Models for Scalable Data Attribution and Valuation [31.732282915190467]
  Inspired by human cognition, we introduce RISE (Readout Influence Sketching Estimator)<n> RISE focuses on influence hotspots at the output layer, where influence signals concentrate, and the gradient admits a decomposed outer-product form.<n>Across the OLMo (1B-32B) and Pythia (14M-6.9B) families, RISE reduces index storage by up to 112$times$ compared to RapidIn.<n>We validate RISE on three tasks: Howdy backdoor data detection, Finance-Medical domain separation, and Brain Rot high-quality data selection.
  arXiv  Detail & Related papers  (2026-04-17T16:07:11Z)
• GIRL: Generative Imagination Reinforcement Learning via Information-Theoretic Hallucination Control [0.0]
  We introduce GIRL (Generative Imagination Reinforcement Learning), a latent world-model framework that addresses this failure mode with two key components.<n> GIRL reduces latent rollout drift by 38 to 61 percent across tasks relative to DreamerV3, improves return, and requires fewer environment interactions on long-horizon tasks.<n>A distilled-prior variant reduces overhead and improves computational efficiency relative to the full model.
  arXiv  Detail & Related papers  (2026-04-08T17:14:21Z)
• Dual-Graph Multi-Agent Reinforcement Learning for Handover Optimization [39.348940144274515]
  We propose a TD3-D-MA in an ns-3 system-level simulator configured with real network operator parameters across heterogeneous traffic regimes.<n>Results show that TD3-D-MA generalizes network throughput overologies standard HO3-Dly.
  arXiv  Detail & Related papers  (2026-03-25T08:48:48Z)
• Sharpness-aware Federated Graph Learning [16.148982247077157]
  One of many impediments to applying graph neural networks (GNNs) to large-scale real-world graph data is the challenge of centralized training.<n> Federated graph learning (FGL) addresses this by enabling collaborative GNN model training without sharing private data.
  arXiv  Detail & Related papers  (2025-12-18T06:57:13Z)
• Continual Action Quality Assessment via Adaptive Manifold-Aligned Graph Regularization [53.82400605816587]
  Action Quality Assessment (AQA) quantifies human actions in videos, supporting applications in sports scoring, rehabilitation, and skill evaluation.<n>A major challenge lies in the non-stationary nature of quality distributions in real-world scenarios.<n>We introduce Continual AQA (CAQA), which equips with Continual Learning capabilities to handle evolving distributions.
  arXiv  Detail & Related papers  (2025-10-08T10:09:47Z)
• CEM-FBGTinyDet: Context-Enhanced Foreground Balance with Gradient Tuning for tiny Objects [2.321156185872456]
  We propose E-FPN-BS, a novel architecture integrating multi-scale feature enhancement and adaptive optimization.<n>First, our Context Enhancement Module(CEM) employs dual-branch processing to align and compress high-level features for effective global-local fusion.<n>Second, the Foreground-Background Separation Module (FBSM) generates spatial gating masks that dynamically amplify discriminative regions.
  arXiv  Detail & Related papers  (2025-06-11T16:13:38Z)
• Learning Robust Heterogeneous Graph Representations via Contrastive-Reconstruction under Sparse Semantics [13.555683316315683]
  Masked autoencoders (MAE) and contrastive learning (CL) are two prominent paradigms in graph self-supervised learning.<n>This paper introduces HetCRF, a novel dual-channel self-supervised learning framework for heterogeneous graphs.<n>HetCRF uses a two-stage aggregation strategy to adapt embedding semantics, making it compatible with both MAE and CL.
  arXiv  Detail & Related papers  (2025-06-07T06:35:42Z)
• ScaleGNN: Towards Scalable Graph Neural Networks via Adaptive High-order Neighboring Feature Fusion [73.85920403511706]
  We propose ScaleGNN, a novel framework that adaptively fuses multi-hop node features for scalable and effective graph learning.<n>We show that ScaleGNN consistently outperforms state-of-the-art GNNs in both predictive accuracy and computational efficiency.
  arXiv  Detail & Related papers  (2025-04-22T14:05:11Z)
• LASE: Learned Adjacency Spectral Embeddings [9.227991604045416]
  We learn nodal Adjacency Spectral Embeddings (ASE) from graph inputs.<n>LASE is interpretable, parameter efficient, robust to inputs with unobserved edges.<n>LASE layers combine Graph Convolutional Network (GCN) and fully-connected Graph Attention Network (GAT) modules.
  arXiv  Detail & Related papers  (2024-12-23T17:35:19Z)
• FedRGL: Robust Federated Graph Learning for Label Noise [5.296582539751589]
  Federated Graph Learning (FGL) is a distributed machine learning paradigm based on graph neural networks.<n>We propose a robust federated graph learning method with label noise, termed FedRGL.<n>We show that FedRGL outperforms 12 baseline methods across various noise rates, types, and numbers of clients.
  arXiv  Detail & Related papers  (2024-11-28T04:37:04Z)
• Degree-Conscious Spiking Graph for Cross-Domain Adaptation [51.58506501415558]
  Spiking Graph Networks (SGNs) have demonstrated significant potential in graph classification.<n>We introduce a novel framework named Degree-Consicious Spiking Graph for Cross-Domain Adaptation (DeSGraDA)<n>DeSGraDA enhances generalization across domains with three key components.
  arXiv  Detail & Related papers  (2024-10-09T13:45:54Z)
• Divide and Contrast: Source-free Domain Adaptation via Adaptive Contrastive Learning [122.62311703151215]
  Divide and Contrast (DaC) aims to connect the good ends of both worlds while bypassing their limitations. DaC divides the target data into source-like and target-specific samples, where either group of samples is treated with tailored goals. We further align the source-like domain with the target-specific samples using a memory bank-based Maximum Mean Discrepancy (MMD) loss to reduce the distribution mismatch.
  arXiv  Detail & Related papers  (2022-11-12T09:21:49Z)
• Local Augmentation for Graph Neural Networks [78.48812244668017]
  We introduce the local augmentation, which enhances node features by its local subgraph structures. Based on the local augmentation, we further design a novel framework: LA-GNN, which can apply to any GNN models in a plug-and-play manner.
  arXiv  Detail & Related papers  (2021-09-08T18:10:08Z)
• Cross-Domain Facial Expression Recognition: A Unified Evaluation Benchmark and Adversarial Graph Learning [85.6386289476598]
  We develop a novel adversarial graph representation adaptation (AGRA) framework for cross-domain holistic-local feature co-adaptation. We conduct extensive and fair evaluations on several popular benchmarks and show that the proposed AGRA framework outperforms previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-08-03T15:00:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.