From Dialogue to Execution: Mixture-of-Agents Assisted Interactive Planning for Behavior Tree-Based Long-Horizon Robot Execution

• URL: http://arxiv.org/abs/2603.01113v1
• Date: Sun, 01 Mar 2026 13:54:51 GMT
• Title: From Dialogue to Execution: Mixture-of-Agents Assisted Interactive Planning for Behavior Tree-Based Long-Horizon Robot Execution
• Abstract summary: Large language models (LLMs) enable robots to generate high-level action plans from natural language instructions.<n>In long-horizon tasks, such approaches often require many questions, increasing user burden.<n>We propose a framework that integrates Mixture-of-Agents (MoA)-based proxy answering into interactive planning.
• Score: 10.513251543236684
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Interactive task planning with large language models (LLMs) enables robots to generate high-level action plans from natural language instructions. However, in long-horizon tasks, such approaches often require many questions, increasing user burden. Moreover, flat plan representations become difficult to manage as task complexity grows. We propose a framework that integrates Mixture-of-Agents (MoA)-based proxy answering into interactive planning and generates Behavior Trees (BTs) for structured long-term execution. The MoA consists of multiple LLM-based expert agents that answer general or domain-specific questions when possible, reducing unnecessary human intervention. The resulting BT hierarchically represents task logic and enables retry mechanisms and dynamic switching among multiple robot policies. Experiments on cocktail-making tasks show that the proposed method reduces human response requirements by approximately 27% while maintaining structural and semantic similarity to fully human-answered BTs. Real-robot experiments on a smoothie-making task further demonstrate successful long-horizon execution with adaptive policy switching and recovery from action failures. These results indicate that MoA-assisted interactive planning improves dialogue efficiency while preserving execution quality in real-world robotic tasks.

Related papers

• A Closed-Loop Multi-Agent Framework for Robust Multi-Robot Manipulation [37.78546236286294]
  Large Language Models (LLMs) offer the reasoning capabilities to decompose objectives into actionable plans.<n>We propose a hierarchical closed-loop agentic framework to ensure robust multi-robot manipulation.<n>Our system consists of three specialized agents: the Planning Agent decomposes instructions into allocated sub-tasks, the Manipulation Agent for each robot executes actions via adaptive tool use, and the Verification Agent closes the loop by monitoring physical outcomes and feeding back semantic corrections.
  arXiv  Detail & Related papers  (2026-07-08T04:23:41Z)
• Reducing Mental Workload through On-Demand Human Assistance for Physical Action Failures in LLM-based Multi-Robot Coordination [6.763690463901024]
  In this study, we propose a human-in-the-loop framework that integrates remote error resolution into multi-robot planning.<n>In this method, robots execute tasks autonomously; however, when an irrecoverable failure occurs, the LLM requests assistance from an operator.<n>For easily collectable items, it achieved task progress equivalent to full remote control.
  arXiv  Detail & Related papers  (2026-03-30T08:21:04Z)
• H-AIM: Orchestrating LLMs, PDDL, and Behavior Trees for Hierarchical Multi-Robot Planning [3.2800662172795114]
  H-AIM is a novel embodied multi-robot task planning framework.<n>It exploits large language models (LLMs) to parse instructions and generate Planning Domain Definition Language (PDDL) problem descriptions.<n>It compiles the resulting plan into behavior trees for reactive control.
  arXiv  Detail & Related papers  (2026-01-16T07:59:50Z)
• RoBridge: A Hierarchical Architecture Bridging Cognition and Execution for General Robotic Manipulation [90.81956345363355]
  RoBridge is a hierarchical intelligent architecture for general robotic manipulation.<n>It consists of a high-level cognitive planner (HCP) based on a large-scale pre-trained vision-language model (VLM)<n>It unleashes the procedural skill of reinforcement learning, effectively bridging the gap between cognition and execution.
  arXiv  Detail & Related papers  (2025-05-03T06:17:18Z)
• REMAC: Self-Reflective and Self-Evolving Multi-Agent Collaboration for Long-Horizon Robot Manipulation [57.628771707989166]
  We propose an adaptive multi-agent planning framework, termed REMAC, that enables efficient, scene-agnostic multi-robot long-horizon task planning and execution.<n>ReMAC incorporates two key modules: a self-reflection module performing pre-conditions and post-condition checks in the loop to evaluate progress and refine plans, and a self-evolvement module dynamically adapting plans based on scene-specific reasoning.
  arXiv  Detail & Related papers  (2025-03-28T03:51:40Z)
• Agent-Oriented Planning in Multi-Agent Systems [54.429028104022066]
  We propose AOP, a novel framework for agent-oriented planning in multi-agent systems.<n>In this study, we identify three critical design principles of agent-oriented planning, including solvability, completeness, and non-redundancy.<n> Extensive experiments demonstrate the advancement of AOP in solving real-world problems compared to both single-agent systems and existing planning strategies for multi-agent systems.
  arXiv  Detail & Related papers  (2024-10-03T04:07:51Z)
• COHERENT: Collaboration of Heterogeneous Multi-Robot System with Large Language Models [49.24666980374751]
  COHERENT is a novel LLM-based task planning framework for collaboration of heterogeneous multi-robot systems.<n>A Proposal-Execution-Feedback-Adjustment mechanism is designed to decompose and assign actions for individual robots.<n>The experimental results show that our work surpasses the previous methods by a large margin in terms of success rate and execution efficiency.
  arXiv  Detail & Related papers  (2024-09-23T15:53:41Z)
• TDAG: A Multi-Agent Framework based on Dynamic Task Decomposition and Agent Generation [41.21899915378596]
  We propose a multi-agent framework based on dynamic Task Decomposition and Agent Generation (TDAG)<n>This framework dynamically decomposes complex tasks into smaller subtasks and assigns each to a specifically generated subagent.<n>ItineraryBench is designed to assess agents' abilities in memory, planning, and tool usage across tasks of varying complexity.
  arXiv  Detail & Related papers  (2024-02-15T18:27:37Z)
• AutoTAMP: Autoregressive Task and Motion Planning with LLMs as Translators and Checkers [20.857692296678632]
  For effective human-robot interaction, robots need to understand, plan, and execute complex, long-horizon tasks. Recent advances in large language models have shown promise for translating natural language into robot action sequences. We show that our approach outperforms several methods using LLMs as planners in complex task domains.
  arXiv  Detail & Related papers  (2023-06-10T21:58:29Z)
• ProgPrompt: Generating Situated Robot Task Plans using Large Language Models [68.57918965060787]
  Large language models (LLMs) can be used to score potential next actions during task planning. We present a programmatic LLM prompt structure that enables plan generation functional across situated environments.
  arXiv  Detail & Related papers  (2022-09-22T20:29:49Z)
• A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Execution [54.385344986265714]
  We propose a persistent spatial semantic representation method to bridge the gap between language and robot actions. We evaluate our approach on the ALFRED benchmark and achieve state-of-the-art results, despite completely avoiding the commonly used step-by-step instructions.
  arXiv  Detail & Related papers  (2021-07-12T17:47:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.