On the Roles of LLMs in Planning: Embedding LLMs into Planning Graphs
- URL: http://arxiv.org/abs/2403.00783v1
- Date: Sun, 18 Feb 2024 15:53:32 GMT
- Title: On the Roles of LLMs in Planning: Embedding LLMs into Planning Graphs
- Authors: Hankz Hankui Zhuo and Xin Chen and Rong Pan
- Abstract summary: We study the insight of the planning capability of large language models (LLMs) in off-the-shelf planning frameworks.
We propose a novel LLMs-based planning framework with LLMs embedded in two levels of planning graphs.
We empirically exhibit the effectiveness of our proposed framework in various planning domains.
- Score: 13.854158637408647
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Plan synthesis aims to generate a course of actions or policies to transit
given initial states to goal states, provided domain models that could be
designed by experts or learnt from training data or interactions with the
world. Intrigued by the claims of emergent planning capabilities in large
language models (LLMs), works have been proposed to investigate the planning
effectiveness of LLMs, without considering any utilization of off-the-shelf
planning techniques in LLMs. In this paper, we aim to further study the insight
of the planning capability of LLMs by investigating the roles of LLMs in
off-the-shelf planning frameworks. To do this, we investigate the effectiveness
of embedding LLMs into one of the well-known planning frameworks, graph-based
planning, proposing a novel LLMs-based planning framework with LLMs embedded in
two levels of planning graphs, i.e., mutual constraints generation level and
constraints solving level. We empirically exhibit the effectiveness of our
proposed framework in various planning domains.
Related papers
- Learning to Plan for Retrieval-Augmented Large Language Models from Knowledge Graphs [59.76268575344119]
We introduce a novel framework for enhancing large language models' (LLMs) planning capabilities by using planning data derived from knowledge graphs (KGs)
LLMs fine-tuned with KG data have improved planning capabilities, better equipping them to handle complex QA tasks that involve retrieval.
arXiv Detail & Related papers (2024-06-20T13:07:38Z) - Exploring and Benchmarking the Planning Capabilities of Large Language Models [57.23454975238014]
We construct a benchmark suite encompassing both classical planning domains and natural language scenarios.
Second, we investigate the use of in-context learning (ICL) to enhance LLM planning, exploring the direct relationship between increased context length and improved planning performance.
Third, we demonstrate the positive impact of fine-tuning LLMs on optimal planning paths, as well as the effectiveness of incorporating model-driven search procedures.
arXiv Detail & Related papers (2024-06-18T22:57:06Z) - Open Grounded Planning: Challenges and Benchmark Construction [44.86307213996181]
We propose a new planning task--open grounded planning.
The primary objective of open grounded planning is to ask the model to generate an executable plan based on a variable action set.
Then we test current state-of-the-art LLMs along with five planning approaches, revealing that existing LLMs and methods still struggle to address the challenges posed by grounded planning in open domains.
arXiv Detail & Related papers (2024-06-05T03:46:52Z) - What's the Plan? Evaluating and Developing Planning-Aware Techniques for Language Models [7.216683826556268]
Large language models (LLMs) are increasingly used for applications that require planning capabilities.
We introduce SimPlan, a novel hybrid-method, and evaluate its performance in a new challenging setup.
arXiv Detail & Related papers (2024-02-18T07:42:49Z) - Understanding the planning of LLM agents: A survey [98.82513390811148]
This survey provides the first systematic view of LLM-based agents planning, covering recent works aiming to improve planning ability.
Comprehensive analyses are conducted for each direction, and further challenges in the field of research are discussed.
arXiv Detail & Related papers (2024-02-05T04:25:24Z) - On the Prospects of Incorporating Large Language Models (LLMs) in
Automated Planning and Scheduling (APS) [23.024862968785147]
This paper investigates eight categories based on the unique applications of LLMs in addressing various aspects of planning problems.
A critical insight resulting from our review is that the true potential of LLMs unfolds when they are integrated with traditional symbolic planners.
arXiv Detail & Related papers (2024-01-04T19:22:09Z) - LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoning [65.86754998249224]
We develop a novel hybrid planner that leverages a conventional rule-based planner in conjunction with an LLM-based planner.
Our approach navigates complex scenarios which existing planners struggle with, produces well-reasoned outputs while also remaining grounded through working alongside the rule-based approach.
arXiv Detail & Related papers (2023-12-30T02:53:45Z) - AutoPlanBench: Automatically generating benchmarks for LLM planners from
PDDL [52.005042190810116]
We present AutoPlanBench, a novel method for automatically converting planning benchmarks written in PDDL into textual descriptions.
We show that while the best LLM planners do well on some planning tasks, others remain out of reach of current methods.
arXiv Detail & Related papers (2023-11-16T11:55:27Z) - On the Planning Abilities of Large Language Models : A Critical
Investigation [34.262740442260515]
We evaluate the effectiveness of LLMs in generating plans autonomously in commonsense planning tasks.
In the LLM-Modulo setting, we demonstrate that LLM-generated plans can improve the search process for underlying sound planners.
arXiv Detail & Related papers (2023-05-25T06:32:23Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.