CoPAL: Corrective Planning of Robot Actions with Large Language Models
- URL: http://arxiv.org/abs/2310.07263v1
- Date: Wed, 11 Oct 2023 07:39:42 GMT
- Title: CoPAL: Corrective Planning of Robot Actions with Large Language Models
- Authors: Frank Joublin, Antonello Ceravola, Pavel Smirnov, Felix Ocker, Joerg
Deigmoeller, Anna Belardinelli, Chao Wang, Stephan Hasler, Daniel Tanneberg,
Michael Gienger
- Abstract summary: We propose a system architecture that orchestrates a seamless interplay between cognitive levels, encompassing reasoning, planning, and motion generation.
At its core lies a novel replanning strategy that handles physically grounded, logical, and semantic errors in the generated plans.
- Score: 8.209152055117283
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In the pursuit of fully autonomous robotic systems capable of taking over
tasks traditionally performed by humans, the complexity of open-world
environments poses a considerable challenge. Addressing this imperative, this
study contributes to the field of Large Language Models (LLMs) applied to task
and motion planning for robots. We propose a system architecture that
orchestrates a seamless interplay between multiple cognitive levels,
encompassing reasoning, planning, and motion generation. At its core lies a
novel replanning strategy that handles physically grounded, logical, and
semantic errors in the generated plans. We demonstrate the efficacy of the
proposed feedback architecture, particularly its impact on executability,
correctness, and time complexity via empirical evaluation in the context of a
simulation and two intricate real-world scenarios: blocks world, barman and
pizza preparation.
Related papers
- Commonsense Reasoning for Legged Robot Adaptation with Vision-Language Models [81.55156507635286]
Legged robots are physically capable of navigating a diverse variety of environments and overcoming a wide range of obstructions.
Current learning methods often struggle with generalization to the long tail of unexpected situations without heavy human supervision.
We propose a system, VLM-Predictive Control (VLM-PC), combining two key components that we find to be crucial for eliciting on-the-fly, adaptive behavior selection.
arXiv Detail & Related papers (2024-07-02T21:00:30Z) - Synergising Human-like Responses and Machine Intelligence for Planning in Disaster Response [10.294618771570985]
We propose an attention-based cognitive architecture inspired by Dual Process Theory (DPT)
This framework integrates, in an online fashion, rapid yet (human-like) responses with the slow but optimized planning capabilities of machine intelligence.
arXiv Detail & Related papers (2024-04-15T15:47:08Z) - RoboScript: Code Generation for Free-Form Manipulation Tasks across Real
and Simulation [77.41969287400977]
This paper presents textbfRobotScript, a platform for a deployable robot manipulation pipeline powered by code generation.
We also present a benchmark for a code generation benchmark for robot manipulation tasks in free-form natural language.
We demonstrate the adaptability of our code generation framework across multiple robot embodiments, including the Franka and UR5 robot arms.
arXiv Detail & Related papers (2024-02-22T15:12:00Z) - Interactive Planning Using Large Language Models for Partially
Observable Robotics Tasks [54.60571399091711]
Large Language Models (LLMs) have achieved impressive results in creating robotic agents for performing open vocabulary tasks.
We present an interactive planning technique for partially observable tasks using LLMs.
arXiv Detail & Related papers (2023-12-11T22:54:44Z) - Transferring Foundation Models for Generalizable Robotic Manipulation [82.12754319808197]
We propose a novel paradigm that effectively leverages language-reasoning segmentation mask generated by internet-scale foundation models.
Our approach can effectively and robustly perceive object pose and enable sample-efficient generalization learning.
Demos can be found in our submitted video, and more comprehensive ones can be found in link1 or link2.
arXiv Detail & Related papers (2023-06-09T07:22:12Z) - Fast Kinodynamic Planning on the Constraint Manifold with Deep Neural
Networks [29.239926645660823]
This paper introduces a novel learning-to-plan framework that exploits the concept of constraint manifold.
Our approach generates plans satisfying an arbitrary set of constraints and computes them in a short constant time, namely the inference time of a neural network.
We validate our approach on two simulated tasks and in a demanding real-world scenario, where we use a Kuka LBR Iiwa 14 robotic arm to perform the hitting movement in robotic Air Hockey.
arXiv Detail & Related papers (2023-01-11T06:54:11Z) - ProgPrompt: Generating Situated Robot Task Plans using Large Language
Models [68.57918965060787]
Large language models (LLMs) can be used to score potential next actions during task planning.
We present a programmatic LLM prompt structure that enables plan generation functional across situated environments.
arXiv Detail & Related papers (2022-09-22T20:29:49Z) - Cooperative Task and Motion Planning for Multi-Arm Assembly Systems [32.56644393804845]
Multi-robot assembly systems are becoming increasingly appealing in manufacturing.
Planning for these systems in a manner that ensures each robot is simultaneously productive, and not idle, is challenging.
We present a task and motion planning framework that jointly plans safe, low-makespan plans for a team of robots to assemble complex spatial structures.
arXiv Detail & Related papers (2022-03-04T18:12:49Z) - NeRP: Neural Rearrangement Planning for Unknown Objects [49.191284597526]
We propose NeRP (Neural Rearrangement Planning), a deep learning based approach for multi-step neural object rearrangement planning.
NeRP works with never-before-seen objects, that is trained on simulation data, and generalizes to the real world.
arXiv Detail & Related papers (2021-06-02T17:56:27Z) - iCORPP: Interleaved Commonsense Reasoning and Probabilistic Planning on
Robots [46.13039152809055]
We present a novel algorithm, called iCORPP, to simultaneously estimate the current world state, reason about world dynamics, and construct task-oriented controllers.
Results show significant improvements in scalability, efficiency, and adaptiveness, compared to competitive baselines.
arXiv Detail & Related papers (2020-04-18T17:46:59Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.