Embodied Multi-Agent Coordination by Aligning World Models Through Dialogue

• URL: http://arxiv.org/abs/2605.12920v2
• Date: Sat, 16 May 2026 03:44:07 GMT
• Title: Embodied Multi-Agent Coordination by Aligning World Models Through Dialogue
• Abstract summary: Communication can bridge this gap by allowing agents to share observations and align their world models.<n>We extend PARTNR, a benchmark for collaborative household robotics, with a natural-language dialogue channel that enables two agents with partial observability to communicate.<n>Our experiments reveal that dialogue reduces action conflicts 40 to 83 percentage points but degrades task success relative to silent coordination.
• Score: 9.790389620810933
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Effective collaboration between embodied agents requires more than acting in a shared environment; it demands communication grounded in each agent's evolving understanding of the world. When agents can only partially observe their surroundings, coordination without communication is provably hard, but communication can, in principle, bridge this gap by allowing agents to share observations and align their world models. In this work, we examine whether LLM-based embodied agents actually realize the ability to communicate. We extend PARTNR, a benchmark for collaborative household robotics, with a natural-language dialogue channel that enables two agents with partial observability to communicate during task execution. To evaluate whether dialogue leads to genuine world-model alignment rather than superficial coordination, we propose a framework for measuring world-model alignment defined over per-agent world graphs: observation convergence (do private world models align over time?), information novelty (do messages convey what the partner lacks?), and belief-sensitive messaging (do agents model what their partner knows?). Our experiments across three LLMs reveal that dialogue reduces action conflicts 40 to 83 percentage points but degrades task success relative to silent coordination. Using our metrics, we characterize the gap between superficial coordination and genuine world-model alignment, and identify where current models fall on this spectrum.

Related papers

• When Numbers Start Talking: Implicit Numerical Coordination Among LLM-Based Agents [45.43445469098021]
  This paper presents a game-theoretic study of covert communication in multi-agent systems.<n>We analyse interactions across four canonical game-theoretic settings under different communication regimes.<n>Considering heterogeneous agent personalities and both one-shot and repeated games, we characterise when covert signals emerge and how they shape coordination and strategic outcomes.
  arXiv  Detail & Related papers  (2026-01-07T12:07:48Z)
• Communication and Verification in LLM Agents towards Collaboration under Information Asymmetry [17.472005826931127]
  This paper studies Large Language Model (LLM) agents in task collaboration.<n>We extend Einstein Puzzles, a symbolic puzzle, to a table-top game.<n> Empirical results highlight the critical importance of aligned communication.
  arXiv  Detail & Related papers  (2025-10-29T15:03:53Z)
• Learning to Interact in World Latent for Team Coordination [53.51290193631586]
  This work presents a novel representation learning framework, interactive world latent (IWoL), to facilitate team coordination in multi-agent reinforcement learning (MARL)<n>Our key insight is to construct a learnable representation space that jointly captures inter-agent relations and task-specific world information by directly modeling communication protocols.<n>Our representation can be used not only as an implicit latent for each agent, but also as an explicit message for communication.
  arXiv  Detail & Related papers  (2025-09-29T22:13:39Z)
• Communicating Plans, Not Percepts: Scalable Multi-Agent Coordination with Embodied World Models [0.0]
  A central question in Multi-Agent Reinforcement Learning (MARL) is whether to engineer communication protocols or learn them end-to-end.<n>We propose and compare two communication strategies for a cooperative task-allocation problem.<n>Our experiments reveal that while emergent communication is viable in simple settings, the engineered, world model-based approach shows superior performance, sample efficiency, and scalability as complexity increases.
  arXiv  Detail & Related papers  (2025-08-04T21:29:07Z)
• Decentralized Collective World Model for Emergent Communication and Coordination [7.039405984678893]
  We propose a fully decentralized multi-agent world model that enables both symbol emergence for communication and coordinated behavior.<n>Our method integrates world models with communication channels, enabling agents to predict environmental dynamics, estimate states from partial observations, and share critical information.
  arXiv  Detail & Related papers  (2025-04-04T11:17:52Z)
• Communication Learning in Multi-Agent Systems from Graph Modeling Perspective [62.13508281188895]
  We introduce a novel approach wherein we conceptualize the communication architecture among agents as a learnable graph. We introduce a temporal gating mechanism for each agent, enabling dynamic decisions on whether to receive shared information at a given time.
  arXiv  Detail & Related papers  (2024-11-01T05:56:51Z)
• Learning Multi-Agent Communication from Graph Modeling Perspective [62.13508281188895]
  We introduce a novel approach wherein we conceptualize the communication architecture among agents as a learnable graph. Our proposed approach, CommFormer, efficiently optimize the communication graph and concurrently refines architectural parameters through gradient descent in an end-to-end manner.
  arXiv  Detail & Related papers  (2024-05-14T12:40:25Z)
• COMBO: Compositional World Models for Embodied Multi-Agent Cooperation [65.46592503910875]
  We investigate the problem of embodied multi-agent cooperation, where decentralized agents must cooperate given only egocentric views of the world.<n>We learn a compositional world model for multi-agent cooperation by factorizing the naturally composable joint actions of multiple agents.<n>We evaluate our methods on three challenging benchmarks with 2-4 agents.
  arXiv  Detail & Related papers  (2024-04-16T17:59:11Z)
• Building Cooperative Embodied Agents Modularly with Large Language Models [104.57849816689559]
  We address challenging multi-agent cooperation problems with decentralized control, raw sensory observations, costly communication, and multi-objective tasks instantiated in various embodied environments. We harness the commonsense knowledge, reasoning ability, language comprehension, and text generation prowess of LLMs and seamlessly incorporate them into a cognitive-inspired modular framework. Our experiments on C-WAH and TDW-MAT demonstrate that CoELA driven by GPT-4 can surpass strong planning-based methods and exhibit emergent effective communication.
  arXiv  Detail & Related papers  (2023-07-05T17:59:27Z)
• Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents [83.52684405389445]
  We introduce the collaborative multi-object navigation task CoMON. In this task, an oracle agent has detailed environment information in the form of a map. It communicates with a navigator agent that perceives the environment visually and is tasked to find a sequence of goals. We show that the emergent communication can be grounded to the agent observations and the spatial structure of the 3D environment.
  arXiv  Detail & Related papers  (2021-10-12T06:56:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.