Decomposing Communication Gain and Delay Cost Under Cross-Timestep Delays in Cooperative Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2604.03785v1
• Date: Sat, 04 Apr 2026 16:14:41 GMT
• Title: Decomposing Communication Gain and Delay Cost Under Cross-Timestep Delays in Cooperative Multi-Agent Reinforcement Learning
• Abstract summary: We formalize a delayed-communication partially observable Markov game (DeComm-POMG)<n>We decompose a message's effect into emphcommunication gain and emphdelay cost, yielding the Communication Gain and Delay Cost metric.
• Score: 1.6735996397275577
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Communication is essential for coordination in \emph{cooperative} multi-agent reinforcement learning under partial observability, yet \emph{cross-timestep} delays cause messages to arrive multiple timesteps after generation, inducing temporal misalignment and making information stale when consumed. We formalize this setting as a delayed-communication partially observable Markov game (DeComm-POMG) and decompose a message's effect into \emph{communication gain} and \emph{delay cost}, yielding the Communication Gain and Delay Cost (CGDC) metric. We further establish a value-loss bound showing that the degradation induced by delayed messages is upper-bounded by a discounted accumulation of an information gap between the action distributions induced by timely versus delayed messages. Guided by CGDC, we propose \textbf{CDCMA}, an actor--critic framework that requests messages only when predicted CGDC is positive, predicts future observations to reduce misalignment at consumption, and fuses delayed messages via CGDC-guided attention. Experiments on no-teammate-vision variants of Cooperative Navigation and Predator Prey, and on SMAC maps across multiple delay levels show consistent improvements in performance, robustness, and generalization, with ablations validating each component.

Related papers

• AsyncCouple-Flow: Asynchronous Cross-Modal Coupling and Flow Matching for Spatio-Temporal Forecasting [14.101995007248997]
  AsyncCouple-Flow supports weathercasting, traffic prediction, and earth-system modeling by combining heterogeneous sources such as physical fields, satellite sensors, and insitu sensors.<n>AsyncCouple-Flow outperforms state-of-the-art baselines and remains robust with up to two missing modalities.
  arXiv  Detail & Related papers  (2026-09-15T03:15:14Z)
• Inference-Time Graph Engineering for Multi-Agent LLM Workflows [51.56038180639833]
  We introduce ReActNet, a training-free framework that compiles a query and a set of role-specialized agents into a sequence of directed communication graphs.<n>We show that ReActNet consistently improves over fixed-topology and learned-topology baselines while maintaining competitive inference cost.
  arXiv  Detail & Related papers  (2026-09-04T23:32:11Z)
• ExComm: Exploration-Stage Communication for Error-Resilient Agentic Test-Time Scaling [57.42714978834704]
  ExComm is a communication protocol for exploration-stage agentic test-time scaling.<n>We show that ExComm consistently outperforms strong test-time scaling baselines.
  arXiv  Detail & Related papers  (2026-05-21T07:38:44Z)
• When Semantic Communication Meets Queueing: Cross-Layer Latency and Task Fidelity Optimization [5.523167414178437]
  SemCom with learned encoder-decoder architectures enables end-to-end learning of compact task-oriented representations optimized for the wireless channel.<n>This paper studies semantic image transmission over block fading with AWGN using a multi-task semantic autoencoder.<n>We develop online semantic-rate controllers that adapt the latent dimension per update under a long-term semantic error constraint.
  arXiv  Detail & Related papers  (2026-05-06T23:30:46Z)
• Monitoring Neural Training with Topology: A Footprint-Predictable Collapse Index [51.56484100374058]
  We propose an online, topology-aware monitor for evolving neural representations that couples Modular Morse Homology Maintenance (MMHM) with a composite Collapse Index (CI)<n>Instead of rebuilding complexes each epoch, we apply sparse edits at a fixed scale and maintain a discrete Morse matching, yielding fast, incremental updates.
  arXiv  Detail & Related papers  (2026-04-28T19:21:21Z)
• Wireless Federated Multi-Task LLM Fine-Tuning via Sparse-and-Orthogonal LoRA [61.12136997430116]
  Decentralized federated learning (DFL) based on low-rank adaptation (LoRA) enables mobile devices with multi-task datasets to collaboratively fine-tune a large language model (LLM) by exchanging locally updated parameters with a subset of neighboring devices via wireless connections for knowledge integration.<n> directly aggregating parameters fine-tuned on heterogeneous datasets induces three primary issues across the DFL life-cycle: (i) catastrophic knowledge forgetting during fine-tuning process, arising from conflicting update directions caused by data heterogeneity; (ii) textitinefficient communication and convergence during model aggregation process,
  arXiv  Detail & Related papers  (2026-02-24T02:45:32Z)
• Cooperation Breakdown in LLM Agents Under Communication Delays [3.619444603816032]
  We propose the FLCOA framework to conceptualize how cooperation and coordination emerge in groups of autonomous agents.<n>To examine the effect of communication delay, we introduce a Continuous Prisoner's Dilemma with Communication Delay.<n>We find that excessive delay reduces cycles of exploitation, yielding a U-shaped relationship between delay magnitude and mutual cooperation.
  arXiv  Detail & Related papers  (2026-02-12T09:31:47Z)
• Bandwidth-constrained Variational Message Encoding for Cooperative Multi-agent Reinforcement Learning [23.866517021196724]
  Graph-based multi-agent reinforcement learning (MARL) enables coordinated behavior under partial observability by modeling agents as nodes and communication links as edges.<n>We study this bandwidth-limited regime and show that naive dimensionality reduction consistently degrades coordination performance.<n>We introduce Band-constrained Variational Message (BVME), a lightweight module that treats messages as samples from Gaussians regularized via KL divergence to an uninformative prior.<n>BVME achieves comparable or superior performance while using 67--83% fewer message dimensions, with gains most pronounced on sparse graphs where message quality critically impacts coordination.
  arXiv  Detail & Related papers  (2025-12-11T23:56:43Z)
• Real-Time Inference for Distributed Multimodal Systems under Communication Delay Uncertainty [37.15356899831919]
  Connected cyber-physical systems perform inference based on real-time inputs from multiple data streams.<n>We propose a novel neuro-inspired non-blocking inference paradigm that employs adaptive temporal windows of integration.<n>Our framework achieves robust real-time inference with finer-grained control over the accuracy-latency tradeoff.
  arXiv  Detail & Related papers  (2025-11-20T10:48:54Z)
• CoDe: Communication Delay-Tolerant Multi-Agent Collaboration via Dual Alignment of Intent and Timeliness [21.627120541083553]
  This paper proposes a novel framework, Communication Delay-tolerant Multi-Agent Collaboration (CoDe)<n>At first, CoDe learns an intent representation as messages through future action inference.<n>Then, CoDe devises a dual alignment mechanism of intent and timeliness to strengthen the fusion process of asynchronous messages.
  arXiv  Detail & Related papers  (2025-01-09T12:57:41Z)
• DASA: Delay-Adaptive Multi-Agent Stochastic Approximation [64.32538247395627]
  We consider a setting in which $N$ agents aim to speedup a common Approximation problem by acting in parallel and communicating with a central server. To mitigate the effect of delays and stragglers, we propose textttDASA, a Delay-Adaptive algorithm for multi-agent Approximation.
  arXiv  Detail & Related papers  (2024-03-25T22:49:56Z)
• DACOM: Learning Delay-Aware Communication for Multi-Agent Reinforcement Learning [19.36041216505116]
  We show that ignoring communication delays has detrimental effects on collaborations. We design a delay-aware multi-agent communication model (DACOM) to adapt communication to delays. Our experiments reveal that DACOM has a non-negligible performance improvement over other mechanisms.
  arXiv  Detail & Related papers  (2022-12-03T14:20:59Z)
• Accelerating Federated Edge Learning via Optimized Probabilistic Device Scheduling [57.271494741212166]
  This paper formulates and solves the communication time minimization problem. It is found that the optimized policy gradually turns its priority from suppressing the remaining communication rounds to reducing per-round latency as the training process evolves. The effectiveness of the proposed scheme is demonstrated via a use case on collaborative 3D objective detection in autonomous driving.
  arXiv  Detail & Related papers  (2021-07-24T11:39:17Z)
• Learning to Communicate and Correct Pose Errors [75.03747122616605]
  We study the setting proposed in V2VNet, where nearby self-driving vehicles jointly perform object detection and motion forecasting in a cooperative manner. We propose a novel neural reasoning framework that learns to communicate, to estimate potential errors, and to reach a consensus about those errors.
  arXiv  Detail & Related papers  (2020-11-10T18:19:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.