Long-horizon Embodied Planning with Implicit Logical Inference and Hallucination Mitigation

• URL: http://arxiv.org/abs/2409.15658v2
• Date: Thu, 13 Mar 2025 10:15:59 GMT
• Title: Long-horizon Embodied Planning with Implicit Logical Inference and Hallucination Mitigation
• Authors: Siyuan Liu, Jiawei Du, Sicheng Xiang, Zibo Wang, Dingsheng Luo,
• Abstract summary: We present ReLEP, a novel framework for Real-time Long-horizon Embodied Planning.<n>ReLEP can complete a wide range of long-horizon tasks without in-context examples by learning implicit logical inference through fine-tuning.
• Score: 7.668848364013772
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-horizon embodied planning underpins embodied AI. To accomplish long-horizon tasks, one of the most feasible ways is to decompose abstract instructions into a sequence of actionable steps. Foundation models still face logical errors and hallucinations in long-horizon planning, unless provided with highly relevant examples to the tasks. However, providing highly relevant examples for any random task is unpractical. Therefore, we present ReLEP, a novel framework for Real-time Long-horizon Embodied Planning. ReLEP can complete a wide range of long-horizon tasks without in-context examples by learning implicit logical inference through fine-tuning. The fine-tuned large vision-language model formulates plans as sequences of skill functions. These functions are selected from a carefully designed skill library. ReLEP is also equipped with a Memory module for plan and status recall, and a Robot Configuration module for versatility across robot types. In addition, we propose a data generation pipeline to tackle dataset scarcity. When constructing the dataset, we considered the implicit logical relationships, enabling the model to learn implicit logical relationships and dispel hallucinations. Through comprehensive evaluations across various long-horizon tasks, ReLEP demonstrates high success rates and compliance to execution even on unseen tasks and outperforms state-of-the-art baseline methods.

Related papers

• Data-Agnostic Robotic Long-Horizon Manipulation with Vision-Language-Guided Closed-Loop Feedback [12.600525101342026]
  We introduce DAHLIA, a data-agnostic framework for language-conditioned long-horizon robotic manipulation. LLMs are large language models for real-time task planning and execution. Our framework demonstrates state-of-the-art performance across diverse long-horizon tasks, achieving strong generalization in both simulated and real-world scenarios.
  arXiv  Detail & Related papers  (2025-03-27T20:32:58Z)
• Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks [36.63527489464188]
  Plan-and-Act is a framework that incorporates explicit planning into large language models (LLMs) Plan-and-Act consists of a Planner model which generates structured, high-level plans to achieve user goals, and an Executor model that translates these plans into environment-specific actions. We present a state-of-the-art 57.58% success rate on the WebArena-Lite benchmark as well as a text-only state-of-the-art 81.36% success rate on WebVoyager.
  arXiv  Detail & Related papers  (2025-03-12T17:40:52Z)
• Plan-over-Graph: Towards Parallelable LLM Agent Schedule [53.834646147919436]
  Large Language Models (LLMs) have demonstrated exceptional abilities in reasoning for task planning. This paper introduces a novel paradigm, plan-over-graph, in which the model first decomposes a real-life textual task into executable subtasks and constructs an abstract task graph. The model then understands this task graph as input and generates a plan for parallel execution.
  arXiv  Detail & Related papers  (2025-02-20T13:47:51Z)
• Diffusion Meets Options: Hierarchical Generative Skill Composition for Temporally-Extended Tasks [12.239868705130178]
  We propose a data-driven hierarchical framework that generates and updates plans based on instruction specified by linear temporal logic (LTL) Our method decomposes temporal tasks into chain of options with hierarchical reinforcement learning from offline non-expert datasets. We devise a determinantal-guided posterior sampling technique during batch generation, which improves the speed and diversity of diffusion generated options.
  arXiv  Detail & Related papers  (2024-10-03T11:10:37Z)
• Unlocking Reasoning Potential in Large Langauge Models by Scaling Code-form Planning [94.76546523689113]
  We introduce CodePlan, a framework that generates and follows textcode-form plans -- pseudocode that outlines high-level, structured reasoning processes. CodePlan effectively captures the rich semantics and control flows inherent to sophisticated reasoning tasks. It achieves a 25.1% relative improvement compared with directly generating responses.
  arXiv  Detail & Related papers  (2024-09-19T04:13:58Z)
• Plan-Seq-Learn: Language Model Guided RL for Solving Long Horizon Robotics Tasks [50.27313829438866]
  Plan-Seq-Learn (PSL) is a modular approach that uses motion planning to bridge the gap between abstract language and learned low-level control. PSL achieves success rates of over 85%, out-performing language-based, classical, and end-to-end approaches.
  arXiv  Detail & Related papers  (2024-05-02T17:59:31Z)
• Learning Logic Specifications for Policy Guidance in POMDPs: an Inductive Logic Programming Approach [57.788675205519986]
  We learn high-quality traces from POMDP executions generated by any solver. We exploit data- and time-efficient Indu Logic Programming (ILP) to generate interpretable belief-based policy specifications. We show that learneds expressed in Answer Set Programming (ASP) yield performance superior to neural networks and similar to optimal handcrafted task-specifics within lower computational time.
  arXiv  Detail & Related papers  (2024-02-29T15:36:01Z)
• RoboGPT: an intelligent agent of making embodied long-term decisions for daily instruction tasks [13.29302304547683]
  Large Language Models (LLMs) in natural language processing have inspired efforts to use LLMs in complex robot planning. We propose a RoboGPT agent for making embodied long-term decisions for daily tasks. The proposed RoboGPT agent outperforms SOTA methods on the ALFRED daily tasks.
  arXiv  Detail & Related papers  (2023-11-27T09:20:23Z)
• Generalizable Long-Horizon Manipulations with Large Language Models [91.740084601715]
  This work introduces a framework harnessing the capabilities of Large Language Models (LLMs) to generate primitive task conditions for generalizable long-horizon manipulations. We create a challenging robotic manipulation task suite based on Pybullet for long-horizon task evaluation.
  arXiv  Detail & Related papers  (2023-10-03T17:59:46Z)
• Compositional Foundation Models for Hierarchical Planning [52.18904315515153]
  We propose a foundation model which leverages expert foundation model trained on language, vision and action data individually together to solve long-horizon tasks. We use a large language model to construct symbolic plans that are grounded in the environment through a large video diffusion model. Generated video plans are then grounded to visual-motor control, through an inverse dynamics model that infers actions from generated videos.
  arXiv  Detail & Related papers  (2023-09-15T17:44:05Z)
• Embodied Task Planning with Large Language Models [86.63533340293361]
  We propose a TAsk Planing Agent (TaPA) in embodied tasks for grounded planning with physical scene constraint. During inference, we discover the objects in the scene by extending open-vocabulary object detectors to multi-view RGB images collected in different achievable locations. Experimental results show that the generated plan from our TaPA framework can achieve higher success rate than LLaVA and GPT-3.5 by a sizable margin.
  arXiv  Detail & Related papers  (2023-07-04T17:58:25Z)
• Learning to Reason over Scene Graphs: A Case Study of Finetuning GPT-2 into a Robot Language Model for Grounded Task Planning [45.51792981370957]
  We investigate the applicability of a smaller class of large language models (LLMs) in robotic task planning by learning to decompose tasks into subgoal specifications for a planner to execute sequentially. Our method grounds the input of the LLM on the domain that is represented as a scene graph, enabling it to translate human requests into executable robot plans. Our findings suggest that the knowledge stored in an LLM can be effectively grounded to perform long-horizon task planning, demonstrating the promising potential for the future application of neuro-symbolic planning methods in robotics.
  arXiv  Detail & Related papers  (2023-05-12T18:14:32Z)
• Efficient Learning of High Level Plans from Play [57.29562823883257]
  We present Efficient Learning of High-Level Plans from Play (ELF-P), a framework for robotic learning that bridges motion planning and deep RL. We demonstrate that ELF-P has significantly better sample efficiency than relevant baselines over multiple realistic manipulation tasks.
  arXiv  Detail & Related papers  (2023-03-16T20:09:47Z)
• A Framework for Neurosymbolic Robot Action Planning using Large Language Models [3.0501524254444767]
  We present a framework aimed at bridging the gap between symbolic task planning and machine learning approaches. The rationale is training Large Language Models (LLMs) into a neurosymbolic task planner compatible with the Planning Domain Definition Language (PDDL) Preliminary results in selected domains show that our method can: (i) solve 95.5% of problems in a test data set of 1,000 samples; (ii) produce plans up to 13.5% shorter than a traditional symbolic planner; (iii) reduce average overall waiting times for a plan availability by up to 61.4%.
  arXiv  Detail & Related papers  (2023-03-01T11:54:22Z)
• ProgPrompt: Generating Situated Robot Task Plans using Large Language Models [68.57918965060787]
  Large language models (LLMs) can be used to score potential next actions during task planning. We present a programmatic LLM prompt structure that enables plan generation functional across situated environments.
  arXiv  Detail & Related papers  (2022-09-22T20:29:49Z)
• Long-Horizon Planning and Execution with Functional Object-Oriented Networks [79.94575713911189]
  We introduce the idea of exploiting object-level knowledge as a FOON for task planning and execution. Our approach automatically transforms FOON into PDDL and leverages off-the-shelf planners, action contexts, and robot skills. We demonstrate our approach on long-horizon tasks in CoppeliaSim and show how learned action contexts can be extended to never-before-seen scenarios.
  arXiv  Detail & Related papers  (2022-07-12T19:29:35Z)
• Hierarchies of Planning and Reinforcement Learning for Robot Navigation [22.08479169489373]
  In many navigation tasks, high-level (HL) task representations, like a rough floor plan, are available. Previous work has demonstrated efficient learning by hierarchal approaches consisting of path planning in the HL representation. This work proposes a novel hierarchical framework that utilizes a trainable planning policy for the HL representation.
  arXiv  Detail & Related papers  (2021-09-23T07:18:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.