CoCoBench: A Cooperative Coordination Benchmark for Embodied Multi-Agent Task Planning

• URL: http://arxiv.org/abs/2608.28266v1
• Date: Fri, 28 Aug 2026 12:29:23 GMT
• Title: CoCoBench: A Cooperative Coordination Benchmark for Embodied Multi-Agent Task Planning
• Abstract summary: CoCoBench is a construct-level benchmark for evaluating multi-agent embodied coordination in executable household tasks.<n>We evaluate 11 leading MLLMs across different coordination modes, observation inputs, and numbers of agents.
• Score: 26.72126160106256
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Agent systems powered by multimodal large language models (MLLMs) have advanced rapidly in recent years, yet existing embodied-agent benchmarks still lack fine-grained diagnostics for multi-agent coordination. Most benchmarks either focus on single-agent task completion or summarize multi-agent behavior with overall task success rates, which can obscure coordination failures such as duplicated work, violations of ordering constraints, resource contention, and desynchronized handoffs. In this paper, we introduce CoCoBench, a construct-level benchmark for evaluating multi-agent embodied coordination in executable household tasks. CoCoBench contains 897 oracle-validated instances organized around four recurring coordination constructs: task allocation, sequential ordering, mutual exclusion, and handoff coordination. In addition to task success rate, CoCoBench provides construct-level scores that measure whether agents coordinate effectively. We evaluate 11 leading MLLMs across different coordination modes, observation inputs, and numbers of agents. The results show that coordination ability is highly construct-specific: strong overall performance does not imply balanced competence across different coordination types. These findings point to new directions for designing targeted model architectures and improving multi-agent coordination ability.

Related papers

• Benchmarking Open-Ended Multi-Agent Coordination in Language Agents [48.1363632826625]
  Alem embeds procedurally generated coordination tasks, soft specialisation, communication, and controllable coordination difficulty into a long-horizon survival world.<n>We introduce $alem$, a JAX-based benchmark for open-ended multi-agent coordination built on Craftax-like dynamics.
  arXiv  Detail & Related papers  (2026-06-06T21:13:43Z)
• When Does Hierarchy Help? Benchmarking Agent Coordination in Event-Driven Industrial Scheduling [7.912877527497326]
  DESBench is a benchmark for evaluating agent coordination in hierarchical event-driven scheduling.<n>We focus on four representative coordination paradigms: centralized, hierarchical, heterarchical, and holonic.
  arXiv  Detail & Related papers  (2026-05-13T08:33:28Z)
• Coordination Matters: Evaluation of Cooperative Multi-Agent Reinforcement Learning [1.9806597528054068]
  We propose a coordination-aware evaluation perspective that supplements return with process-level diagnostics.<n>We instantiate this perspective using STAT, a controlled commitment-constrained spatial task-allocation testbed.<n>Our results show that similar return trends can reflect distinct coordination mechanisms, including differences in redundant assignment, assignment diversity, and task-completion efficiency.
  arXiv  Detail & Related papers  (2026-05-07T16:50:53Z)
• Improving the Efficiency of Language Agent Teams with Adaptive Task Graphs [52.26652574704317]
  Large language models (LLMs) are increasingly deployed in teams, yet existing coordination approaches often occupy two extremes.<n>We introduce Language Agent Teams for Task Evolution (LATTE), a framework for coordinating LLM teams inspired by distributed systems.
  arXiv  Detail & Related papers  (2026-05-07T14:19:17Z)
• OrgAgent: Organize Your Multi-Agent System like a Company [75.47076168155817]
  We introduce OrgAgent, a company-style hierarchical multi-agent framework.<n>We find that multi-agent systems organized in a company-style hierarchy generally outperform other organizational structures.
  arXiv  Detail & Related papers  (2026-04-01T15:21:14Z)
• MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents [59.825725526176655]
  Large Language Models (LLMs) have shown remarkable capabilities as autonomous agents.<n>Existing benchmarks either focus on single-agent tasks or are confined to narrow domains, failing to capture the dynamics of multi-agent coordination and competition.<n>We introduce MultiAgentBench, a benchmark designed to evaluate LLM-based multi-agent systems across diverse, interactive scenarios.
  arXiv  Detail & Related papers  (2025-03-03T05:18:50Z)
• CaPo: Cooperative Plan Optimization for Efficient Embodied Multi-Agent Cooperation [98.11670473661587]
  CaPo improves cooperation efficiency with two phases: 1) meta-plan generation, and 2) progress-adaptive meta-plan and execution.<n> Experimental results on the ThreeDworld Multi-Agent Transport and Communicative Watch-And-Help tasks demonstrate that CaPo achieves much higher task completion rate and efficiency compared with state-of-the-arts.
  arXiv  Detail & Related papers  (2024-11-07T13:08:04Z)
• RACA: Relation-Aware Credit Assignment for Ad-Hoc Cooperation in Multi-Agent Deep Reinforcement Learning [55.55009081609396]
  We propose a novel method, called Relation-Aware Credit Assignment (RACA), which achieves zero-shot generalization in ad-hoc cooperation scenarios. RACA takes advantage of a graph-based encoder relation to encode the topological structure between agents. Our method outperforms baseline methods on the StarCraftII micromanagement benchmark and ad-hoc cooperation scenarios.
  arXiv  Detail & Related papers  (2022-06-02T03:39:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.