CoEnv: Driving Embodied Multi-Agent Collaboration via Compositional Environment

• URL: http://arxiv.org/abs/2604.05484v1
• Date: Tue, 07 Apr 2026 06:24:41 GMT
• Title: CoEnv: Driving Embodied Multi-Agent Collaboration via Compositional Environment
• Abstract summary: CoEnv is a framework that leverages simulation for safe strategy exploration while ensuring reliable real-world deployment.<n>CoEnv operates through three stages: real-to-sim scene reconstruction that digitizes physical workspaces, VLM-driven action synthesis supporting both real-time planning with code-based trajectory generation, and validated sim-to-real transfer with collision detection for safe deployment.
• Score: 54.84167973913561
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-agent embodied systems hold promise for complex collaborative manipulation, yet face critical challenges in spatial coordination, temporal reasoning, and shared workspace awareness. Inspired by human collaboration where cognitive planning occurs separately from physical execution, we introduce the concept of compositional environment -- a synergistic integration of real-world and simulation components that enables multiple robotic agents to perceive intentions and operate within a unified decision-making space. Building on this concept, we present CoEnv, a framework that leverages simulation for safe strategy exploration while ensuring reliable real-world deployment. CoEnv operates through three stages: real-to-sim scene reconstruction that digitizes physical workspaces, VLM-driven action synthesis supporting both real-time planning with high-level interfaces and iterative planning with code-based trajectory generation, and validated sim-to-real transfer with collision detection for safe deployment. Extensive experiments on challenging multi-arm manipulation benchmarks demonstrate CoEnv's effectiveness in achieving high task success rates and execution efficiency, establishing a new paradigm for multi-agent embodied AI.

Related papers

• Cooperative Multi-UAV Navigation in Complex Environments via Systematic Multi-Agent Deep Reinforcement Learning [14.620999662200795]
  Local optima traps, sparse rewards, learning imbalance among agents, and insufficient cross-scenario generalisation.<n>This paper proposes a multi-agent deep reinforcement learning framework that addresses these issues through coordinated exploration, demonstration exploitation, safe curriculum scheduling, and structure-aware generalisation.<n> Simulation results confirm strong performance in collaboration success rate, navigation robustness, zero-shot cross-scenario generalisation, and dynamic environment adaptability.
  arXiv  Detail & Related papers  (2026-07-28T14:16:11Z)
• Exploratory, Communicative, and Deployable: Vision-Driven Embodied Agents for Open-World Mobile Manipulation [47.58516950734307]
  Existing frameworks often rely on privileged simulator states or assume complete instructions, bypassing realistic deployment challenges.<n>We present REAL, an agentic framework for open-world mobile manipulation.<n>We design diverse task compositions to drive data collection, supervised fine-tuning, and online reinforcement learning.
  arXiv  Detail & Related papers  (2026-07-15T09:55:45Z)
• SynAgent: Generalizable Cooperative Humanoid Manipulation via Solo-to-Cooperative Agent Synergy [59.86199649493784]
  Controllable cooperative humanoid manipulation is a fundamental yet challenging problem for embodied intelligence.<n>We present SynAgent, a framework that enables scalable and physically plausible cooperative manipulation.<n>In experiments, SynAgent significantly outperforms existing baselines in both cooperative imitation and trajectory-conditioned control.
  arXiv  Detail & Related papers  (2026-04-20T17:46:20Z)
• TeamHOI: Learning a Unified Policy for Cooperative Human-Object Interactions with Any Team Size [54.0714652192002]
  Physics-based humanoid control has achieved remarkable progress in enabling realistic and high-performing single-agent behaviors.<n>We present TeamHOI, a framework that enables a single decentralized policy to handle cooperative HOIs across any number of cooperating agents.
  arXiv  Detail & Related papers  (2026-03-09T05:52:13Z)
• Collaborative Planning with Concurrent Synchronization for Operationally Constrained UAV-UGV Teams [12.49123582162583]
  Collaborative Planning with Concurrent Synchronization (CoPCS) is a learning-based approach that integrates a heterogeneous graph transformer for operationally constrained task encoding with a transformer decoder for joint, synchronized co-planning.<n>Our method provides the novel multi-robot capability of synchronized concurrent co-planning and substantially improves team performance.
  arXiv  Detail & Related papers  (2026-03-06T21:36:25Z)
• EmCoop: A Framework and Benchmark for Embodied Cooperation Among LLM Agents [18.802912315746564]
  EmCoop is a benchmark framework for studying cooperation in embodied multi-agent systems.<n>Our framework separates a high-level cognitive layer from a low-level embodied interaction layer.<n>We propose generalizable, process-level metrics that diagnose collaboration quality and failure modes.
  arXiv  Detail & Related papers  (2026-02-27T22:28:33Z)
• Efficient and Compliant Control Framework for Versatile Human-Humanoid Collaborative Transportation [2.4087148947930634]
  We present a control framework that enables humanoid robots to perform collaborative transportation tasks with a human partner.<n>The framework supports both translational and rotational motions, which are fundamental to co-transport scenarios.<n>It comprises three components: a high-level planner, a low-level controller, and a stiffness modulation mechanism.
  arXiv  Detail & Related papers  (2025-12-08T18:52:10Z)
• DR. WELL: Dynamic Reasoning and Learning with Symbolic World Model for Embodied LLM-Based Multi-Agent Collaboration [18.250351934101463]
  DR. WELL is a decentralized neurosymbolic framework for cooperative multi-agent planning.<n>Agents propose candidate roles and commit to a joint allocation under consensus and environment constraints.<n>Each agent independently generates and executes a symbolic plan for its role without revealing detailed trajectories.
  arXiv  Detail & Related papers  (2025-11-06T18:37:18Z)
• SynWorld: Virtual Scenario Synthesis for Agentic Action Knowledge Refinement [81.30121762971473]
  SynWorld is a framework that allows agents to autonomously explore environments, optimize, and enhance their understanding of actions.<n>Our experiments demonstrate that SynWorld is an effective and general approach to learning action knowledge in new environments.
  arXiv  Detail & Related papers  (2025-04-04T16:10:57Z)
• Unified Human-Scene Interaction via Prompted Chain-of-Contacts [61.87652569413429]
  Human-Scene Interaction (HSI) is a vital component of fields like embodied AI and virtual reality. This paper presents a unified HSI framework, UniHSI, which supports unified control of diverse interactions through language commands.
  arXiv  Detail & Related papers  (2023-09-14T17:59:49Z)
• Nonprehensile Riemannian Motion Predictive Control [57.295751294224765]
  We introduce a novel Real-to-Sim reward analysis technique to reliably imagine and predict the outcome of taking possible actions for a real robotic platform. We produce a closed-loop controller to reactively push objects in a continuous action space. We observe that RMPC is robust in cluttered as well as occluded environments and outperforms the baselines.
  arXiv  Detail & Related papers  (2021-11-15T18:50:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.