CRAFT: Grounded Multi-Agent Coordination Under Partial Information

• URL: http://arxiv.org/abs/2603.25268v1
• Date: Thu, 26 Mar 2026 10:06:39 GMT
• Title: CRAFT: Grounded Multi-Agent Coordination Under Partial Information
• Abstract summary: CRAFT is a benchmark for evaluating pragmatic communication in large language models.<n>We formalize this problem as a multi-sender pragmatic reasoning task.<n>We find that stronger reasoning ability does not reliably translate to better coordination.
• Score: 10.532912977406811
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce CRAFT, a multi-agent benchmark for evaluating pragmatic communication in large language models under strict partial information. In this setting, multiple agents with complementary but incomplete views must coordinate through natural language to construct a shared 3D structure that no single agent can fully observe. We formalize this problem as a multi-sender pragmatic reasoning task and provide a diagnostic framework that decomposes failures into spatial grounding, belief modeling and pragmatic communication errors, including a taxonomy of behavioral failure profiles in both frontier and open-weight models. Across a diverse set of models, including 8 open-weight and 7 frontier including reasoning models, we find that stronger reasoning ability does not reliably translate to better coordination: smaller open-weight models often match or outperform frontier systems, and improved individual communication does not guarantee successful collaboration. These results suggest that multi-agent coordination remains a fundamentally unsolved challenge for current language models. Our code can be found at https://github.com/csu-signal/CRAFT

Related papers

• Heterogeneity-Aware Belief Synchronization for Semantic Communication in AI-Native 6G Networks [19.957323858362148]
  6G networks will not be serving as communication infrastructures only; rather, they are expected to evolve into intelligent systems.<n>This article presents a heterogeneous-aware belief synchronization framework for 6G AI-native networks.<n>We validate the framework through a case study on a multi-layered terrestrial/non-terrestrial network.
  arXiv  Detail & Related papers  (2026-08-13T15:53:15Z)
• MT-PingEval: Evaluating Multi-Turn Collaboration with Private Information Games [70.37904949359938]
  We evaluate language models in multi-turn interactions using a suite of collaborative games that require effective communication about private information.<n>We find that language models are unable to use interactive collaboration to improve over the non-interactive baseline scenario.<n>We analyze the linguistic features of these dialogues, assessing the roles of sycophancy, information density, and discourse coherence.
  arXiv  Detail & Related papers  (2026-02-27T17:13:20Z)
• Every Step Counts: Decoding Trajectories as Authorship Fingerprints of dLLMs [63.82840470917859]
  We show that the decoding mechanism of dLLMs can be used as a powerful tool for model attribution.<n>We propose a novel information extraction scheme called the Directed Decoding Map (DDM), which captures structural relationships between decoding steps and better reveals model-specific behaviors.
  arXiv  Detail & Related papers  (2025-10-02T06:25:10Z)
• Reasoning-Aware Prompt Orchestration: A Foundation Model for Multi-Agent Language Model Coordination [0.0]
  We present a theoretically-grounded framework for dynamic prompt orchestration that enhances reasoning across multiple specialized agents.<n>This framework addresses three core challenges: logical consistency preservation during agent transitions, reasoning-aware prompt adaptation, and scalable coordination of distributed inference.<n> Experimental results on 1,000 synthetic multi-agent conversations demonstrate a 42% reduction in reasoning latency, a 23% improvement in logical consistency measured by ROUGE-L score, and an 89% success rate for task completion without context loss.
  arXiv  Detail & Related papers  (2025-09-30T22:33:01Z)
• Communicating Plans, Not Percepts: Scalable Multi-Agent Coordination with Embodied World Models [0.0]
  A central question in Multi-Agent Reinforcement Learning (MARL) is whether to engineer communication protocols or learn them end-to-end.<n>We propose and compare two communication strategies for a cooperative task-allocation problem.<n>Our experiments reveal that while emergent communication is viable in simple settings, the engineered, world model-based approach shows superior performance, sample efficiency, and scalability as complexity increases.
  arXiv  Detail & Related papers  (2025-08-04T21:29:07Z)
• Why do AI agents communicate in human language? [6.784986834853486]
  Large Language Models (LLMs) have become foundational to modern AI agent systems.<n>In most existing systems, inter-agent communication relies primarily on natural language.<n>We argue that this design introduces fundamental limitations in agent-to-agent coordination.
  arXiv  Detail & Related papers  (2025-06-03T10:53:29Z)
• Detect, Explain, Escalate: Low-Carbon Dialogue Breakdown Management for LLM-Powered Agents [30.13634341221476]
  Large Language Models (LLMs) are transforming numerous applications, but their susceptibility to conversational breakdowns remains a critical challenge undermining user trust.<n>This paper introduces a "Detect, Explain, Escalate" framework to manage dialogue breakdowns in LLM-powered agents, emphasizing low-carbon operation.
  arXiv  Detail & Related papers  (2025-04-26T07:51:05Z)
• Unlocking the Potential of Model Merging for Low-Resource Languages [66.7716891808697]
  Adapting large language models to new languages typically involves continual pre-training (CT) followed by supervised fine-tuning (SFT) We propose model merging as an alternative for low-resource languages, combining models with distinct capabilities into a single model without additional training. Experiments based on Llama-2-7B demonstrate that model merging effectively endows LLMs for low-resource languages with task-solving abilities, outperforming CT-then-SFT in scenarios with extremely scarce data.
  arXiv  Detail & Related papers  (2024-07-04T15:14:17Z)
• Mitigating Data Imbalance and Representation Degeneration in Multilingual Machine Translation [103.90963418039473]
  Bi-ACL is a framework that uses only target-side monolingual data and a bilingual dictionary to improve the performance of the MNMT model. We show that Bi-ACL is more effective both in long-tail languages and in high-resource languages.
  arXiv  Detail & Related papers  (2023-05-22T07:31:08Z)
• Grounded Decoding: Guiding Text Generation with Grounded Models for Embodied Agents [111.15288256221764]
  Grounded-decoding project aims to solve complex, long-horizon tasks in a robotic setting by leveraging the knowledge of both models. We frame this as a problem similar to probabilistic filtering: decode a sequence that both has high probability under the language model and high probability under a set of grounded model objectives. We demonstrate how such grounded models can be obtained across three simulation and real-world domains, and that the proposed decoding strategy is able to solve complex, long-horizon tasks in a robotic setting by leveraging the knowledge of both models.
  arXiv  Detail & Related papers  (2023-03-01T22:58:50Z)
• RethinkCWS: Is Chinese Word Segmentation a Solved Task? [81.11161697133095]
  The performance of the Chinese Word (CWS) systems has gradually reached a plateau with the rapid development of deep neural networks. In this paper, we take stock of what we have achieved and rethink what's left in the CWS task.
  arXiv  Detail & Related papers  (2020-11-13T11:07:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.