Automating the Generation of Prompts for LLM-based Action Choice in PDDL Planning

• URL: http://arxiv.org/abs/2311.09830v3
• Date: Mon, 06 Jan 2025 13:43:09 GMT
• Title: Automating the Generation of Prompts for LLM-based Action Choice in PDDL Planning
• Authors: Katharina Stein, Daniel Fišer, Jörg Hoffmann, Alexander Koller,
• Abstract summary: Large language models (LLMs) have revolutionized a large variety of NLP tasks. We show how to leverage an LLM to automatically generate NL prompts from PDDL input.
• Score: 59.543858889996024
• License:
• Abstract: Large language models (LLMs) have revolutionized a large variety of NLP tasks. An active debate is to what extent they can do reasoning and planning. Prior work has assessed the latter in the specific context of PDDL planning, based on manually converting three PDDL domains into natural language (NL) prompts. Here we automate this conversion step, showing how to leverage an LLM to automatically generate NL prompts from PDDL input. Our automatically generated NL prompts result in similar LLM-planning performance as the previous manually generated ones. Beyond this, the automation enables us to run much larger experiments, providing for the first time a broad evaluation of LLM planning performance in PDDL.

Related papers

• LLM-AutoDiff: Auto-Differentiate Any LLM Workflow [58.56731133392544]
  We introduce LLM-AutoDiff: a novel framework for Automatic Prompt Engineering (APE) LLMs-AutoDiff treats each textual input as a trainable parameter and uses a frozen backward engine to generate feedback-akin to textual gradients. It consistently outperforms existing textual gradient baselines in both accuracy and training cost.
  arXiv  Detail & Related papers  (2025-01-28T03:18:48Z)
• Leveraging Environment Interaction for Automated PDDL Translation and Planning with Large Language Models [7.3238629831871735]
  Large Language Models (LLMs) have shown remarkable performance in various natural language tasks. Planning problems into the Planning Domain Definition Language (PDDL) has been proposed as a potential solution. We propose a novel approach that leverages LLMs and environment feedback to automatically generate PDDL domain and problem description files.
  arXiv  Detail & Related papers  (2024-07-17T19:50:51Z)
• Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planning [53.6472920229013]
  Large Language Models (LLMs) have demonstrated impressive capability in many natural language tasks. LLMs are prone to produce errors, hallucinations and inconsistent statements when performing multi-step reasoning. We introduce Q*, a framework for guiding LLMs decoding process with deliberative planning.
  arXiv  Detail & Related papers  (2024-06-20T13:08:09Z)
• RePrompt: Planning by Automatic Prompt Engineering for Large Language Models Agents [27.807695570974644]
  We propose a novel method, textscRePrompt, which does agradient descent"-like approach to optimize the step-by-step instructions in the prompts given to LLM agents. By leveraging intermediate feedback, textscRePrompt can optimize the prompt without the need for a final solution checker.
  arXiv  Detail & Related papers  (2024-06-17T01:23:11Z)
• NL2Plan: Robust LLM-Driven Planning from Minimal Text Descriptions [8.004470925893957]
  We present NL2Plan, the first domain-agnostic offline LLM-driven planning system. We evaluate NL2Plan on four planning domains and find that it solves 10 out of 15 tasks. In addition to using NL2Plan in end-to-end mode, users can inspect and correct all of its intermediate results.
  arXiv  Detail & Related papers  (2024-05-07T11:27:13Z)
• Beyond Natural Language: LLMs Leveraging Alternative Formats for Enhanced Reasoning and Communication [79.79948834910579]
  Natural language (NL) has long been the predominant format for human cognition and communication. In this work, we challenge the default use of NL by exploring the utility of non-NL formats in different contexts.
  arXiv  Detail & Related papers  (2024-02-28T16:07:54Z)
• TIC: Translate-Infer-Compile for accurate "text to plan" using LLMs and Logical Representations [0.0]
  We study the problem of generating plans for given natural language planning task requests. Our approach comprises of (a) translate: using an LLM only for generating a interpretable intermediate representation of natural language task description. We observe that using an LLM to only output the intermediate representation significantly reduces LLM errors.
  arXiv  Detail & Related papers  (2024-02-09T18:39:13Z)
• Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models [52.98743860365194]
  We propose a new fine-tuning method called Self-Play fIne-tuNing (SPIN) At the heart of SPIN lies a self-play mechanism, where the LLM refines its capability by playing against instances of itself. This sheds light on the promise of self-play, enabling the achievement of human-level performance in LLMs without the need for expert opponents.
  arXiv  Detail & Related papers  (2024-01-02T18:53:13Z)
• Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning [39.29964085305846]
  Methods that use pre-trained large language models directly as planners are currently impractical due to limited correctness of plans. In this work, we introduce a novel alternative paradigm that constructs an explicit world (domain) model in planning domain definition language (PDDL) and then uses it to plan with sound domain-independent planners.
  arXiv  Detail & Related papers  (2023-05-24T08:59:15Z)
• Guiding Large Language Models via Directional Stimulus Prompting [114.84930073977672]
  We introduce Directional Stimulus Prompting, a novel framework for guiding black-box large language models (LLMs) toward specific desired outputs. Instead of directly adjusting LLMs, our method employs a small tunable policy model to generate an auxiliary directional stimulus prompt for each input instance.
  arXiv  Detail & Related papers  (2023-02-22T17:44:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.