HEART: Coordination of Heterogeneous Expert Agents for Physically Grounded Robotic Task Planning

• URL: http://arxiv.org/abs/2606.25404v1
• Date: Wed, 24 Jun 2026 05:02:37 GMT
• Title: HEART: Coordination of Heterogeneous Expert Agents for Physically Grounded Robotic Task Planning
• Abstract summary: Large Language Models (LLMs) can reason over complex instructions but often fail to satisfy the physical and spatial constraints required for robotic task planning.<n>We present a heterogeneous multi-LLM framework that decomposes instructions into atomic reasoning tasks.<n>We show that HEART consistently improves plan success compared to single-LLM and rule-based planners.
• Score: 13.14097890573283
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Large Language Models (LLMs) can reason over complex instructions but often fail to satisfy the physical and spatial constraints required for robotic task planning. Recent LLM-based planners directly translate text into action sequences, yet they lack structured reasoning about feasibility, reachability, and logical order, resulting in invalid or incomplete plans. We present a heterogeneous multi-LLM framework that decomposes instructions into atomic reasoning tasks and allocates them to role-specialized expert agents under a token budget for real-world computational and communicational constraints. By combining role-oriented reasoning from heterogeneous agents followed by constraint-driven plan synthesis, HEART validates capability, reachability, and constraint conditions before planning and helps produce physically executable plans while maintaining efficiency. Experiments across different household benchmarks show that HEART consistently improves plan success compared to single-LLM and rule-based planners, demonstrating that heterogeneous LLM collaboration enables robust and scalable robotic task planning under resource constraints.

Related papers

• EASy: Towards Efficient LLM-Based Agentic System [54.10819421625103]
  We propose EASy, a trainable agentic framework that jointly optimize task performance and computational efficiency through reinforcement learning.<n> EASy equips an LLM-based orchestrator with explicit knowledge of the capability and cost profiles of heterogeneous executors, enabling context-sensitive coordination beyond performance-only routing.<n>It further introduces a milestone-plan-act workflow that decomposes complex tasks into manageable milestones, constructs dependency-aware execution graphs, assigns suitable executors, and parallelizes independent steps while adapting subsequent decisions to intermediate outcomes.
  arXiv  Detail & Related papers  (2026-08-05T08:50:36Z)
• AdaPlanBench: Evaluating Adaptive Planning in Large Language Model Agents under World and User Constraints [65.27107190916512]
  We introduce AdaPlanBench, an interactive benchmark for evaluating whether Large Language Model (LLM) agents can adaptively plan and re-plan under progressively revealed constraints.<n>AdaPlanBench is built on 307 household tasks, with a scalable constraint construction pipeline that augments each task with dual constraints.<n>Experiments show that adaptive planning under dual constraints remains challenging, with the best model reaching only 67.75% accuracy.
  arXiv  Detail & Related papers  (2026-06-04T02:47:29Z)
• Melding LLM and temporal logic for reliable human-swarm collaboration in complex scenarios [17.56189537207367]
  We introduce a neuro-symbolic framework for long-horizon human-swarm collaboration.<n>We formalize mission goals and operational rules as temporal logic formulas and admissible task orderings as task automata.<n>An event-triggered interaction protocol limits operator involvement to sparse, high-level confirmation and guidance.
  arXiv  Detail & Related papers  (2026-05-08T15:32:14Z)
• From Coarse to Fine: Self-Adaptive Hierarchical Planning for LLM Agents [24.79854509473032]
  Large language model-based agents have emerged as powerful approaches for solving dynamic and multi-step tasks.<n>We propose textbfAdaPlan-H, a self-adaptive hierarchical planning mechanism that mimics human planning strategies.<n>Our method significantly improves task execution success rates while mitigating overplanning at the planning level.
  arXiv  Detail & Related papers  (2026-04-25T07:54:23Z)
• Scale-Plan: Scalable Language-Enabled Task Planning for Heterogeneous Multi-Robot Teams [11.964765256346857]
  Long-horizon task planning for heterogeneous multi-robot systems is essential for deploying collaborative teams in real-world environments.<n>We present Scale-Plan, a scalable LLM-assisted framework that generates compact, task-relevant problem representations from natural language instructions.<n>By filtering irrelevant information prior to planning, Scale-Plan enables efficient decomposition, allocation, and long-horizon plan generation.
  arXiv  Detail & Related papers  (2026-03-09T18:13:18Z)
• HyperTree Planning: Enhancing LLM Reasoning via Hierarchical Thinking [109.09735490692202]
  We propose HyperTree Planning (HTP), a novel reasoning paradigm that constructs hypertree-structured planning outlines for effective planning.<n> Experiments demonstrate the effectiveness of HTP, achieving state-of-the-art accuracy on the TravelPlanner benchmark with Gemini-1.5-Pro, resulting in a 3.6 times performance improvement over o1-preview.
  arXiv  Detail & Related papers  (2025-05-05T02:38:58Z)
• Hierarchical Planning for Complex Tasks with Knowledge Graph-RAG and Symbolic Verification [5.727096041675994]
  Large Language Models (LLMs) have shown promise as robotic planners but often struggle with long-horizon and complex tasks.<n>We propose a neuro-symbolic approach that enhances LLMs-based planners with Knowledge Graph-based RAG for hierarchical plan generation.
  arXiv  Detail & Related papers  (2025-04-06T18:36:30Z)
• Interactive and Expressive Code-Augmented Planning with Large Language Models [62.799579304821826]
  Large Language Models (LLMs) demonstrate strong abilities in common-sense reasoning and interactive decision-making. Recent techniques have sought to structure LLM outputs using control flow and other code-adjacent techniques to improve planning performance. We propose REPL-Plan, an LLM planning approach that is fully code-expressive and dynamic.
  arXiv  Detail & Related papers  (2024-11-21T04:23:17Z)
• Unlocking Reasoning Potential in Large Langauge Models by Scaling Code-form Planning [94.76546523689113]
  We introduce CodePlan, a framework that generates and follows textcode-form plans -- pseudocode that outlines high-level, structured reasoning processes. CodePlan effectively captures the rich semantics and control flows inherent to sophisticated reasoning tasks. It achieves a 25.1% relative improvement compared with directly generating responses.
  arXiv  Detail & Related papers  (2024-09-19T04:13:58Z)
• Unified Task and Motion Planning using Object-centric Abstractions of Motion Constraints [56.283944756315066]
  We propose an alternative TAMP approach that unifies task and motion planning into a single search. Our approach is based on an object-centric abstraction of motion constraints that permits leveraging the computational efficiency of off-the-shelf AI search to yield physically feasible plans.
  arXiv  Detail & Related papers  (2023-12-29T14:00:20Z)
• Conformal Temporal Logic Planning using Large Language Models [26.793426646439073]
  We consider missions that require accomplishing multiple high-level sub-tasks expressed in natural language (NL), in a temporal and logical order.<n>Our goal is to design plans, defined as sequences of robot actions, accomplishing-NL tasks.<n>We propose HERACLEs, a hierarchical neuro-symbolic planner that relies on a novel integration of existing symbolic planners.
  arXiv  Detail & Related papers  (2023-09-18T19:05:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.