SCoUT: Scalable Communication via Utility-Guided Temporal Grouping in Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2603.04833v1
• Date: Thu, 05 Mar 2026 05:33:28 GMT
• Title: SCoUT: Scalable Communication via Utility-Guided Temporal Grouping in Multi-Agent Reinforcement Learning
• Abstract summary: Communication can improve coordination in partially observed multi-agent reinforcement learning (MARL)<n>We introduce textbfSCoUT (textbfScalable textbfCommunication via textbfUtility-guided textbfTemporal grouping)<n>Agents are trained with a three-headed policy: environment action, send decision, and recipient selection.
• Score: 2.064923532131528
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Communication can improve coordination in partially observed multi-agent reinforcement learning (MARL), but learning \emph{when} and \emph{who} to communicate with requires choosing among many possible sender-recipient pairs, and the effect of any single message on future reward is hard to isolate. We introduce \textbf{SCoUT} (\textbf{S}calable \textbf{Co}mmunication via \textbf{U}tility-guided \textbf{T}emporal grouping), which addresses both these challenges via temporal and agent abstraction within traditional MARL. During training, SCoUT resamples \textit{soft} agent groups every \(K\) environment steps (macro-steps) via Gumbel-Softmax; these groups are latent clusters that induce an affinity used as a differentiable prior over recipients. Using the same assignments, a group-aware critic predicts values for each agent group and maps them to per-agent baselines through the same soft assignments, reducing critic complexity and variance. Each agent is trained with a three-headed policy: environment action, send decision, and recipient selection. To obtain precise communication learning signals, we derive counterfactual communication advantages by analytically removing each sender's contribution from the recipient's aggregated messages. This counterfactual computation enables precise credit assignment for both send and recipient-selection decisions. At execution time, all centralized training components are discarded and only the per-agent policy is run, preserving decentralized execution. Project website, videos and code: \hyperlink{https://scout-comm.github.io/}{https://scout-comm.github.io/}

Related papers

• Inference-Time Graph Engineering for Multi-Agent LLM Workflows [51.56038180639833]
  We introduce ReActNet, a training-free framework that compiles a query and a set of role-specialized agents into a sequence of directed communication graphs.<n>We show that ReActNet consistently improves over fixed-topology and learned-topology baselines while maintaining competitive inference cost.
  arXiv  Detail & Related papers  (2026-09-04T23:32:11Z)
• HiComm: Hierarchical Communication for Multi-agent Reinforcement Learning [27.564924459355424]
  We propose textscHiComm, a plug-in communication module that grounds messages in the sender's hierarchical observation.<n>textscHiComm is receiver-driven, and the hierarchy is resolved through a three-stage decoding process.<n>Experiments show that textscHiComm matches or outperforms representative learned communication baselines while reducing communication volume by up to $23times$ per receiver per episode.
  arXiv  Detail & Related papers  (2026-06-28T00:29:12Z)
• Good Agentic Friends Do Not Just Give Verbal Advice: They Can Update Your Weights [58.27172968382275]
  Multi-agent LLM systems usually collaborate by exchanging natural-language messages.<n>We study an alternative communication interface: instead of appending a sender's message to the receiver's context, compile the sender's hidden states into a transient, receiver-specific weight perturbation.
  arXiv  Detail & Related papers  (2026-05-13T17:58:32Z)
• Multi-Agent Decision-Focused Learning via Value-Aware Sequential Communication [0.0]
  We introduce textbfSeqComm-DFL, unifying the sequential communication with decision-focused learning for task performance.<n>Our approach features emphvalue-aware message generation with sequential Stackelberg conditioning.<n>On collaborative healthcare and StarCraft Multi-Agent Challenge benchmarks, SeqComm-DFL achieves four to six times higher cumulative rewards and over 13% win rate improvements.
  arXiv  Detail & Related papers  (2026-04-10T04:23:29Z)
• Agent Q-Mix: Selecting the Right Action for LLM Multi-Agent Systems through Reinforcement Learning [68.85408801740228]
  We propose textbfAgent Q-Mix, a reinforcement learning framework that reformulates topology selection as a cooperative Multi-Agent Reinforcement Learning problem.<n>Our method learns decentralized communication decisions using Qmix value factorization, where each agent selects from a set of communication actions that jointly induce a round-wise communication graph.<n>Agent Q-Mix achieves the highest average accuracy compared to existing methods while demonstrating superior token efficiency and robustness against agent failure.
  arXiv  Detail & Related papers  (2026-04-01T00:38:24Z)
• GoAgent: Group-of-Agents Communication Topology Generation for LLM-based Multi-Agent Systems [55.12339141172908]
  GoAgent is a communication topology generation method that explicitly treats collaborative groups as the atomic units of MAS construction.<n>Experiments on six benchmarks demonstrate the state-of-the-art performance of GoAgent with 93.84% average accuracy while reducing token consumption by about 17%.
  arXiv  Detail & Related papers  (2026-03-20T06:21:32Z)
• DyTopo: Dynamic Topology Routing for Multi-Agent Reasoning via Semantic Matching [15.07152520738373]
  We introduce DyTopo, a manager-guided multi-agent framework that reconstructs a sparse directed communication graph at each round.<n>Conditioned on the manager's round goal, each agent outputs lightweight natural-language query (need) and key (offer) descriptors.<n>DyTopo embeds these descriptors and performs semantic matching, routing private messages only along the induced edges.
  arXiv  Detail & Related papers  (2026-02-05T18:59:51Z)
• FedMGP: Personalized Federated Learning with Multi-Group Text-Visual Prompts [31.907894865146385]
  FedMGP is a new paradigm for personalized federated prompt learning in vision-language models.<n>A diversity loss is introduced to drive each prompt group to specialize in distinct and complementary semantic aspects.<n>FedMGP consistently outperforms prior approaches in both personalization and domain generalization.
  arXiv  Detail & Related papers  (2025-11-01T10:15:04Z)
• Cooperative Pseudo Labeling for Unsupervised Federated Classification [62.9387841396335]
  Unsupervised Federated Learning (UFL) aims to collaboratively train a global model across distributed clients without sharing data or accessing label information.<n>We propose a novel method, underlinetextbfFederated underlinetextbfCooperative underlinetextbfPseudo underlinetextbfLabeling (textbfFedCoPL)<n>In particular, visual prompts containing general image features are aggregated at the server, while text prompts encoding personalized knowledge are retained locally.
  arXiv  Detail & Related papers  (2025-10-11T08:18:26Z)
• AnyMAC: Cascading Flexible Multi-Agent Collaboration via Next-Agent Prediction [77.62279834617475]
  We propose a new framework that rethinks multi-agent coordination through a sequential structure rather than a graph structure.<n>Our method focuses on two key directions: (1) Next-Agent Prediction, which selects the most suitable agent role at each step, and (2) Next-Context Selection, which enables each agent to selectively access relevant information from any previous step.
  arXiv  Detail & Related papers  (2025-06-21T18:34:43Z)
• Cohort Squeeze: Beyond a Single Communication Round per Cohort in Cross-Device Federated Learning [51.560590617691005]
  We investigate whether it is possible to squeeze more juice" out of each cohort than what is possible in a single communication round. Our approach leads to up to 74% reduction in the total communication cost needed to train a FL model in the cross-device setting.
  arXiv  Detail & Related papers  (2024-06-03T08:48:49Z)
• Federated Combinatorial Multi-Agent Multi-Armed Bandits [79.1700188160944]
  This paper introduces a federated learning framework tailored for online optimization with bandit. In this setting, agents subsets of arms, observe noisy rewards for these subsets without accessing individual arm information, and can cooperate and share information at specific intervals.
  arXiv  Detail & Related papers  (2024-05-09T17:40:09Z)
• Federated Contextual Cascading Bandits with Asynchronous Communication and Heterogeneous Users [95.77678166036561]
  We propose a UCB-type algorithm with delicate communication protocols. We give sub-linear regret bounds on par with those achieved in the synchronous framework. Empirical evaluation on synthetic and real-world datasets validates our algorithm's superior performance in terms of regrets and communication costs.
  arXiv  Detail & Related papers  (2024-02-26T05:31:14Z)
• Context-aware Communication for Multi-agent Reinforcement Learning [6.109127175562235]
  We develop a context-aware communication scheme for multi-agent reinforcement learning (MARL) In the first stage, agents exchange coarse representations in a broadcast fashion, providing context for the second stage. Following this, agents utilize attention mechanisms in the second stage to selectively generate messages personalized for the receivers. To evaluate the effectiveness of CACOM, we integrate it with both actor-critic and value-based MARL algorithms.
  arXiv  Detail & Related papers  (2023-12-25T03:33:08Z)
• A Multi-Token Coordinate Descent Method for Semi-Decentralized Vertical Federated Learning [24.60603310894048]
  Communication efficiency is a major challenge in learning (FL) We propose Multi-Token Coordinate Descent (MTCD) MTCD is a tunable communication-efficient for semi-decentralized vertical federation setups.
  arXiv  Detail & Related papers  (2023-09-18T17:59:01Z)
• RGMComm: Return Gap Minimization via Discrete Communications in Multi-Agent Reinforcement Learning [33.86277578441437]
  Communication is crucial for solving cooperative Multi-Agent Reinforcement Learning tasks in partially observable Markov Decision Processes. We propose the Return-Gap-Minimization Communication (RGMComm) algorithm, which is a surprisingly simple design of discrete message generation functions. Evaluations show that RGMComm significantly outperforms state-of-the-art multi-agent communication baselines.
  arXiv  Detail & Related papers  (2023-08-07T07:26:55Z)
• Cooperative Multi-Agent Reinforcement Learning: Asynchronous Communication and Linear Function Approximation [77.09836892653176]
  We study multi-agent reinforcement learning in the setting of episodic Markov decision processes. We propose a provably efficient algorithm based on value that enable asynchronous communication. We show that a minimal $Omega(dM)$ communication complexity is required to improve the performance through collaboration.
  arXiv  Detail & Related papers  (2023-05-10T20:29:29Z)
• A Simple and Provably Efficient Algorithm for Asynchronous Federated Contextual Linear Bandits [77.09836892653176]
  We study federated contextual linear bandits, where $M$ agents cooperate with each other to solve a global contextual linear bandit problem with the help of a central server. We consider the asynchronous setting, where all agents work independently and the communication between one agent and the server will not trigger other agents' communication. We prove that the regret of textttFedLinUCB is bounded by $tildeO(dsqrtsum_m=1M T_m)$ and the communication complexity is $tildeO(dM
  arXiv  Detail & Related papers  (2022-07-07T06:16:19Z)
• Quasi-Equivalence Discovery for Zero-Shot Emergent Communication [63.175848843466845]
  We present a novel problem setting and the Quasi-Equivalence Discovery algorithm that allows for zero-shot coordination (ZSC) We show that these two factors lead to unique optimal ZSC policies in referential games. QED can iteratively discover the symmetries in this setting and converges to the optimal ZSC policy.
  arXiv  Detail & Related papers  (2021-03-14T23:42:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.