RePlan-Bot: Multi-Level Replanning for Embodied Instruction Following

• URL: http://arxiv.org/abs/2605.25851v1
• Date: Mon, 25 May 2026 13:44:18 GMT
• Title: RePlan-Bot: Multi-Level Replanning for Embodied Instruction Following
• Abstract summary: Embodied instruction following (EIF) requires agents to understand and execute complex natural language commands within interactive 3D environments.<n>We introduce RePlan-Bot, a novel EIF agent that performs multi-level, continuous replanning throughout task execution.
• Score: 39.9780803736704
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Embodied instruction following (EIF) requires agents to understand and execute complex natural language commands within interactive 3D environments. Despite recent advances, existing methods often fail in long-horizon planning and handling irreversible state changes, resulting in low task success rates. To address these challenges, we introduce RePlan-Bot, a novel EIF agent that performs multi-level, continuous replanning throughout task execution. RePlan-Bot integrates a high-level LLM-based auditor for dynamic sub-goal adjustments guided by environmental feedback, a commonsense-guided search mechanism based on a multi-layered instance map for precise and structured object localization, and a lightweight ViT-based corrector to preemptively fix risky low-level actions. Evaluated on the ALFRED benchmark, RePlan-Bot achieves state-of-the-art performance in both seen and unseen environments, demonstrating superior adaptability and reliability.

Related papers

• AdaPlanBench: Evaluating Adaptive Planning in Large Language Model Agents under World and User Constraints [65.27107190916512]
  We introduce AdaPlanBench, an interactive benchmark for evaluating whether Large Language Model (LLM) agents can adaptively plan and re-plan under progressively revealed constraints.<n>AdaPlanBench is built on 307 household tasks, with a scalable constraint construction pipeline that augments each task with dual constraints.<n>Experiments show that adaptive planning under dual constraints remains challenging, with the best model reaching only 67.75% accuracy.
  arXiv  Detail & Related papers  (2026-06-04T02:47:29Z)
• Navigating the Clutter: Waypoint-Based Bi-Level Planning for Multi-Robot Systems [54.887871365121775]
  Multi-robot control in cluttered environments is a challenging problem that involves complex physical constraints.<n>We propose a hybrid multi-robot control framework that jointly optimize task and motion planning.<n>Our approach consistently improves task success over motion-agnostic and VLA-based baselines.
  arXiv  Detail & Related papers  (2026-04-22T22:58:47Z)
• ReCAP: Recursive Context-Aware Reasoning and Planning for Large Language Model Agents [61.51091799997476]
  We introduce ReCAP (Recursive Context-Aware Reasoning and Planning), a hierarchical framework with shared context for reasoning and planning in large language models (LLMs)<n>ReCAP combines three key mechanisms: plan-ahead decomposition, structured re-injection of parent plans, and memory-efficient execution.<n>Experiments demonstrate that ReCAP substantially improves subgoal alignment and success rates on various long-horizon reasoning benchmarks.
  arXiv  Detail & Related papers  (2025-10-27T20:03:55Z)
• HiPlan: Hierarchical Planning for LLM-Based Agents with Adaptive Global-Local Guidance [11.621973074884002]
  HiPlan is a hierarchical planning framework for large language model (LLM)-based agents.<n>It decomposes complex tasks into milestone action guides for general direction and step-wise hints for detailed actions.<n>In the offline phase, we construct a milestone library from expert demonstrations, enabling structured experience reuse.<n>In the execution phase, trajectory segments from past milestones are dynamically adapted to generate step-wise hints.
  arXiv  Detail & Related papers  (2025-08-26T14:37:48Z)
• Grounding Language Models with Semantic Digital Twins for Robotic Planning [6.474368392218828]
  We introduce a novel framework that integrates Semantic Digital Twins (SDTs) with Large Language Models (LLMs)<n>The proposed framework effectively combines high-level reasoning with semantic environment understanding, achieving reliable task completion in the face of uncertainty and failure.
  arXiv  Detail & Related papers  (2025-06-19T17:38:00Z)
• LangPert: Detecting and Handling Task-level Perturbations for Robust Object Rearrangement [21.236557779562794]
  LangPert is a language-based framework designed to detect and mitigate Task-Level Perturbations (TLP)<n>LangPert integrates a Visual Language Model (VLM) to comprehensively monitor policy's skill execution and environmental TLP.<n>Our experimental results demonstrate that LangPert handles diverse TLP situations more effectively than baseline methods.
  arXiv  Detail & Related papers  (2025-04-14T05:39:15Z)
• Embodied Long Horizon Manipulation with Closed-loop Code Generation and Incremental Few-shot Adaptation [12.077740860502878]
  Embodied long-horizon manipulation requires robotic systems to process multimodal inputs-such as vision and natural language-and translate them into executable actions.<n>Recent methods have explored using large language models (LLMs) as high-level planners that decompose tasks into subtasks using natural language and guide pretrained low-level controllers.<n>Our framework achieves state-of-the-art performance on 30+ diverse seen and unseen long-horizon tasks across LoHoRavens, CALVIN, Franka Kitchen, and cluttered real-world settings.
  arXiv  Detail & Related papers  (2025-03-27T20:32:58Z)
• AdaPlanner: Adaptive Planning from Feedback with Language Models [56.367020818139665]
  Large language models (LLMs) have recently demonstrated the potential in acting as autonomous agents for sequential decision-making tasks. We propose a closed-loop approach, AdaPlanner, which allows the LLM agent to refine its self-generated plan adaptively in response to environmental feedback. To mitigate hallucination, we develop a code-style LLM prompt structure that facilitates plan generation across a variety of tasks, environments, and agent capabilities.
  arXiv  Detail & Related papers  (2023-05-26T05:52:27Z)
• Plan, Eliminate, and Track -- Language Models are Good Teachers for Embodied Agents [99.17668730578586]
  Pre-trained large language models (LLMs) capture procedural knowledge about the world. Plan, Eliminate, and Track (PET) framework translates a task description into a list of high-level sub-tasks. PET framework leads to a significant 15% improvement over SOTA for generalization to human goal specifications.
  arXiv  Detail & Related papers  (2023-05-03T20:11:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.