Coordination Matters: Evaluation of Cooperative Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2605.06557v1
• Date: Thu, 07 May 2026 16:50:53 GMT
• Title: Coordination Matters: Evaluation of Cooperative Multi-Agent Reinforcement Learning
• Abstract summary: We propose a coordination-aware evaluation perspective that supplements return with process-level diagnostics.<n>We instantiate this perspective using STAT, a controlled commitment-constrained spatial task-allocation testbed.<n>Our results show that similar return trends can reflect distinct coordination mechanisms, including differences in redundant assignment, assignment diversity, and task-completion efficiency.
• Score: 1.9806597528054068
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Cooperative multi-agent reinforcement learning (MARL) benchmarks commonly emphasize aggregate outcomes such as return, success rate, or completion time. While essential, these metrics often fail to reveal how agents coordinate, particularly in settings where agents, tasks, and joint assignment choices scale combinatorially. We propose a coordination-aware evaluation perspective that supplements return with process-level diagnostics. We instantiate this perspective using STAT, a controlled commitment-constrained spatial task-allocation testbed that systematically varies agents, tasks, and environment size while holding observation access and task rules fixed. We evaluate six representative value-based MARL methods across varying levels of centralization. Our results show that similar return trends can reflect distinct coordination mechanisms, including differences in redundant assignment, assignment diversity, and task-completion efficiency. We find that in commitment-constrained task allocation, performance under scale is shaped not only by nominal action-space size, but also by assignment pressure, sparse decision opportunities, and redundant choices among interdependent agents. Our findings motivate coordination-aware evaluation as a necessary complement to return-based benchmarking for cooperative MARL.

Related papers

• CoCoBench: A Cooperative Coordination Benchmark for Embodied Multi-Agent Task Planning [26.72126160106256]
  CoCoBench is a construct-level benchmark for evaluating multi-agent embodied coordination in executable household tasks.<n>We evaluate 11 leading MLLMs across different coordination modes, observation inputs, and numbers of agents.
  arXiv  Detail & Related papers  (2026-08-28T12:29:23Z)
• When Does Hierarchy Help? Benchmarking Agent Coordination in Event-Driven Industrial Scheduling [7.912877527497326]
  DESBench is a benchmark for evaluating agent coordination in hierarchical event-driven scheduling.<n>We focus on four representative coordination paradigms: centralized, hierarchical, heterarchical, and holonic.
  arXiv  Detail & Related papers  (2026-05-13T08:33:28Z)
• Adaptive Value Decomposition: Coordinating a Varying Number of Agents in Urban Systems [19.19146852846605]
  Adaptive Value Decomposition (AVD) is a cooperative MARL framework that adapts to a dynamically changing agent population.<n>A training-execution strategy is designed to accommodate asynchronous decision-making when some agents act at different times.
  arXiv  Detail & Related papers  (2026-02-10T03:41:14Z)
• Multi-agent In-context Coordination via Decentralized Memory Retrieval [39.106914463842685]
  Large transformer models, trained on diverse datasets, have demonstrated impressive few-shot performance on previously unseen tasks.<n>In cooperative Multi-Agent Reinforcement Learning (MARL), where agents must coordinate toward a shared goal, decentralized policy deployment can lead to mismatches in task alignment and reward assignment.<n>We introduce Multi-agent In-context Coordination via Decentralized Memory Retrieval (MAICC), a novel approach designed to enhance coordination by fast adaptation.
  arXiv  Detail & Related papers  (2025-11-13T07:08:31Z)
• Fully Decentralized Cooperative Multi-Agent Reinforcement Learning is A Context Modeling Problem [26.317044969022277]
  This paper studies fully decentralized cooperative multi-agent reinforcement learning, where each agent solely observes the states, its local actions, and the shared rewards.<n>The inability to access other agents' actions often leads to non-stationarity during value function updates and relative overgeneralization during value function estimation.<n>We propose a novel method named Dynamics-Aware Context (DAC), which formalizes the task, as locally perceived by each agent, as an Contextual Markov Decision Process.
  arXiv  Detail & Related papers  (2025-09-19T01:52:44Z)
• Offline Multi-agent Reinforcement Learning via Score Decomposition [51.23590397383217]
  offline cooperative multi-agent reinforcement learning (MARL) faces unique challenges due to distributional shifts.<n>This work is the first work to explicitly address the distributional gap between offline and online MARL.
  arXiv  Detail & Related papers  (2025-05-09T11:42:31Z)
• MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents [59.825725526176655]
  Large Language Models (LLMs) have shown remarkable capabilities as autonomous agents.<n>Existing benchmarks either focus on single-agent tasks or are confined to narrow domains, failing to capture the dynamics of multi-agent coordination and competition.<n>We introduce MultiAgentBench, a benchmark designed to evaluate LLM-based multi-agent systems across diverse, interactive scenarios.
  arXiv  Detail & Related papers  (2025-03-03T05:18:50Z)
• Learning Reward Machines in Cooperative Multi-Agent Tasks [75.79805204646428]
  This paper presents a novel approach to Multi-Agent Reinforcement Learning (MARL) It combines cooperative task decomposition with the learning of reward machines (RMs) encoding the structure of the sub-tasks. The proposed method helps deal with the non-Markovian nature of the rewards in partially observable environments.
  arXiv  Detail & Related papers  (2023-03-24T15:12:28Z)
• RACA: Relation-Aware Credit Assignment for Ad-Hoc Cooperation in Multi-Agent Deep Reinforcement Learning [55.55009081609396]
  We propose a novel method, called Relation-Aware Credit Assignment (RACA), which achieves zero-shot generalization in ad-hoc cooperation scenarios. RACA takes advantage of a graph-based encoder relation to encode the topological structure between agents. Our method outperforms baseline methods on the StarCraftII micromanagement benchmark and ad-hoc cooperation scenarios.
  arXiv  Detail & Related papers  (2022-06-02T03:39:27Z)
• Modeling the Interaction between Agents in Cooperative Multi-Agent Reinforcement Learning [2.9360071145551068]
  We propose a novel cooperative MARL algorithm named as interactive actor-critic(IAC) IAC models the interaction of agents from perspectives of policy and value function. We extend the value decomposition methods to continuous control tasks and evaluate IAC on benchmark tasks including classic control and multi-agent particle environments.
  arXiv  Detail & Related papers  (2021-02-10T01:58:28Z)
• UneVEn: Universal Value Exploration for Multi-Agent Reinforcement Learning [53.73686229912562]
  We propose a novel MARL approach called Universal Value Exploration (UneVEn) UneVEn learns a set of related tasks simultaneously with a linear decomposition of universal successor features. Empirical results on a set of exploration games, challenging cooperative predator-prey tasks requiring significant coordination among agents, and StarCraft II micromanagement benchmarks show that UneVEn can solve tasks where other state-of-the-art MARL methods fail.
  arXiv  Detail & Related papers  (2020-10-06T19:08:47Z)
• Randomized Entity-wise Factorization for Multi-Agent Reinforcement Learning [59.62721526353915]
  Multi-agent settings in the real world often involve tasks with varying types and quantities of agents and non-agent entities. Our method aims to leverage these commonalities by asking the question: What is the expected utility of each agent when only considering a randomly selected sub-group of its observed entities?''
  arXiv  Detail & Related papers  (2020-06-07T18:28:41Z)
• Robust Learning Through Cross-Task Consistency [92.42534246652062]
  We propose a broadly applicable and fully computational method for augmenting learning with Cross-Task Consistency. We observe that learning with cross-task consistency leads to more accurate predictions and better generalization to out-of-distribution inputs.
  arXiv  Detail & Related papers  (2020-06-07T09:24:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.