Interactively Robot Action Planning with Uncertainty Analysis and Active Questioning by Large Language Model

• URL: http://arxiv.org/abs/2308.15684v2
• Date: Wed, 18 Oct 2023 13:31:49 GMT
• Title: Interactively Robot Action Planning with Uncertainty Analysis and Active Questioning by Large Language Model
• Authors: Kazuki Hori, Kanata Suzuki, Tetsuya Ogata
• Abstract summary: The Large Language Model (LLM) to robot action planning has been actively studied. The instructions given to the LLM by natural language may include ambiguity and lack of information depending on the task context. We propose an interactive robot action planning method that allows the LLM to analyze and gather missing information by asking questions to humans.
• Score: 6.695536752781623
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The application of the Large Language Model (LLM) to robot action planning has been actively studied. The instructions given to the LLM by natural language may include ambiguity and lack of information depending on the task context. It is possible to adjust the output of LLM by making the instruction input more detailed; however, the design cost is high. In this paper, we propose the interactive robot action planning method that allows the LLM to analyze and gather missing information by asking questions to humans. The method can minimize the design cost of generating precise robot instructions. We demonstrated the effectiveness of our method through concrete examples in cooking tasks. However, our experiments also revealed challenges in robot action planning with LLM, such as asking unimportant questions and assuming crucial information without asking. Shedding light on these issues provides valuable insights for future research on utilizing LLM for robotics.

Related papers

• LLaRA: Supercharging Robot Learning Data for Vision-Language Policy [56.505551117094534]
  Large Language Models (LLMs) equipped with extensive world knowledge and strong reasoning skills can tackle diverse tasks across domains. We propose LLaRA: Large Language and Robotics Assistant, a framework which formulates robot action policy as conversations.
  arXiv  Detail & Related papers  (2024-06-28T17:59:12Z)
• CLMASP: Coupling Large Language Models with Answer Set Programming for Robotic Task Planning [9.544073786800706]
  Large Language Models (LLMs) possess extensive foundational knowledge and moderate reasoning abilities. It is challenging to ground a LLM-generated plan to be executable for the specified robot with certain restrictions. This paper introduces CLMASP, an approach that couples LLMs with Answer Set Programming (ASP) to overcome the limitations.
  arXiv  Detail & Related papers  (2024-06-05T15:21:44Z)
• Large Language Models for Robotics: Opportunities, Challenges, and Perspectives [46.57277568357048]
  Large language models (LLMs) have undergone significant expansion and have been increasingly integrated across various domains. For embodied tasks, where robots interact with complex environments, text-only LLMs often face challenges due to a lack of compatibility with robotic visual perception. We propose a framework that utilizes multimodal GPT-4V to enhance embodied task planning through the combination of natural language instructions and robot visual perceptions.
  arXiv  Detail & Related papers  (2024-01-09T03:22:16Z)
• Interactive Planning Using Large Language Models for Partially Observable Robotics Tasks [54.60571399091711]
  Large Language Models (LLMs) have achieved impressive results in creating robotic agents for performing open vocabulary tasks. We present an interactive planning technique for partially observable tasks using LLMs.
  arXiv  Detail & Related papers  (2023-12-11T22:54:44Z)
• Language to Rewards for Robotic Skill Synthesis [37.21434094015743]
  We introduce a new paradigm that harnesses large language models (LLMs) to define reward parameters that can be optimized and accomplish variety of robotic tasks. Using reward as the intermediate interface generated by LLMs, we can effectively bridge the gap between high-level language instructions or corrections to low-level robot actions.
  arXiv  Detail & Related papers  (2023-06-14T17:27:10Z)
• AlphaBlock: Embodied Finetuning for Vision-Language Reasoning in Robot Manipulation [50.737355245505334]
  We propose a novel framework for learning high-level cognitive capabilities in robot manipulation tasks. The resulting dataset AlphaBlock consists of 35 comprehensive high-level tasks of multi-step text plans and paired observation.
  arXiv  Detail & Related papers  (2023-05-30T09:54:20Z)
• Learning to Plan with Natural Language [111.76828049344839]
  Large Language Models (LLMs) have shown remarkable performance in various basic natural language tasks. For completing the complex task, we still need a plan for the task to guide LLMs to generate the specific solutions step by step. We propose the Learning to Plan method, which involves two phases: (1) In the first learning task plan phase, it iteratively updates the task plan with new step-by-step solutions and behavioral instructions, which are obtained by prompting LLMs to derive from training error feedback.
  arXiv  Detail & Related papers  (2023-04-20T17:09:12Z)
• ProgPrompt: Generating Situated Robot Task Plans using Large Language Models [68.57918965060787]
  Large language models (LLMs) can be used to score potential next actions during task planning. We present a programmatic LLM prompt structure that enables plan generation functional across situated environments.
  arXiv  Detail & Related papers  (2022-09-22T20:29:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.