Addressing the Challenges of Planning Language Generation

• URL: http://arxiv.org/abs/2505.14763v1
• Date: Tue, 20 May 2025 17:25:23 GMT
• Title: Addressing the Challenges of Planning Language Generation
• Authors: Prabhu Prakash Kagitha, Andrew Zhu, Li Zhang,
• Abstract summary: We design and evaluate 8 different PDDL generation pipelines with open-source models under 50 billion parameters.<n>We find that intuitive approaches such as using a high-resource language wrapper or constrained decoding with grammar decrease performance, yet inference-time scaling approaches such as revision with feedback from the solver and plan validator more than double the performance.
• Score: 6.209697341255856
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Using LLMs to generate formal planning languages such as PDDL that invokes symbolic solvers to deterministically derive plans has been shown to outperform generating plans directly. While this success has been limited to closed-sourced models or particular LLM pipelines, we design and evaluate 8 different PDDL generation pipelines with open-source models under 50 billion parameters previously shown to be incapable of this task. We find that intuitive approaches such as using a high-resource language wrapper or constrained decoding with grammar decrease performance, yet inference-time scaling approaches such as revision with feedback from the solver and plan validator more than double the performance.

Related papers

• Scaling LLM Planning: NL2FLOW for Parametric Problem Generation and Rigorous Evaluation [0.0]
  NL2Flow is a fully automated system for parametrically generating planning problems.<n>It generates a dataset of 2296 low-difficulty problems in automated workflow generation.<n>It evaluates multiple open-sourced, instruct-tuned LLMs without task-specific optimization or architectural modifications.
  arXiv  Detail & Related papers  (2025-07-03T03:02:49Z)
• Learning to Reason and Navigate: Parameter Efficient Action Planning with Large Language Models [63.765846080050906]
  This paper proposes a novel parameter-efficient action planner using large language models (PEAP-LLM) to generate a single-step instruction at each location.<n>Experiments show the superiority of our proposed model on REVERIE compared to the previous state-of-the-art.
  arXiv  Detail & Related papers  (2025-05-12T12:38:20Z)
• An Extensive Evaluation of PDDL Capabilities in off-the-shelf LLMs [11.998185452551878]
  Large language models (LLMs) have exhibited proficiency in code generation and chain-of-thought reasoning.<n>This study evaluates the potential of LLMs to understand and generate Planning Domain Definition Language (PDDL)
  arXiv  Detail & Related papers  (2025-02-27T15:13:07Z)
• Generating Symbolic World Models via Test-time Scaling of Large Language Models [28.258707611580643]
  Planning Domain Definition Language (PDDL) is leveraged as a planning abstraction that enables precise and formal state descriptions.<n>We introduce a simple yet effective algorithm, which first employs a Best-of-N sampling approach to improve the quality of the initial solution.<n>Our method outperforms o1-mini by a considerable margin in the generation of PDDL domains.
  arXiv  Detail & Related papers  (2025-02-07T07:52:25Z)
• On the Limit of Language Models as Planning Formalizers [4.145422873316857]
  Large Language Models fail to create verifiable plans in grounded environments.<n>An emerging line of work shows success in using LLM as a formalizer to generate a formal representation of the planning domain.<n>We observe that large enough models can effectively formalize descriptions as PDDL, outperforming those directly generating plans.
  arXiv  Detail & Related papers  (2024-12-13T05:50:22Z)
• Interactive and Expressive Code-Augmented Planning with Large Language Models [62.799579304821826]
  Large Language Models (LLMs) demonstrate strong abilities in common-sense reasoning and interactive decision-making. Recent techniques have sought to structure LLM outputs using control flow and other code-adjacent techniques to improve planning performance. We propose REPL-Plan, an LLM planning approach that is fully code-expressive and dynamic.
  arXiv  Detail & Related papers  (2024-11-21T04:23:17Z)
• Exploring and Benchmarking the Planning Capabilities of Large Language Models [57.23454975238014]
  This work lays the foundations for improving planning capabilities of large language models (LLMs) We construct a comprehensive benchmark suite encompassing both classical planning benchmarks and natural language scenarios. We investigate the use of many-shot in-context learning to enhance LLM planning, exploring the relationship between increased context length and improved planning performance.
  arXiv  Detail & Related papers  (2024-06-18T22:57:06Z)
• From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems [59.40480894948944]
  Large language model (LLM) empowered agents are able to solve decision-making problems in the physical world. Under this model, the LLM Planner navigates a partially observable Markov decision process (POMDP) by iteratively generating language-based subgoals via prompting. We prove that the pretrained LLM Planner effectively performs Bayesian aggregated imitation learning (BAIL) through in-context learning.
  arXiv  Detail & Related papers  (2024-05-30T09:42:54Z)
• Tree-Planner: Efficient Close-loop Task Planning with Large Language Models [63.06270302774049]
  Tree-Planner reframes task planning with Large Language Models into three distinct phases. Tree-Planner achieves state-of-the-art performance while maintaining high efficiency.
  arXiv  Detail & Related papers  (2023-10-12T17:59:50Z)
• AdaPlanner: Adaptive Planning from Feedback with Language Models [56.367020818139665]
  Large language models (LLMs) have recently demonstrated the potential in acting as autonomous agents for sequential decision-making tasks. We propose a closed-loop approach, AdaPlanner, which allows the LLM agent to refine its self-generated plan adaptively in response to environmental feedback. To mitigate hallucination, we develop a code-style LLM prompt structure that facilitates plan generation across a variety of tasks, environments, and agent capabilities.
  arXiv  Detail & Related papers  (2023-05-26T05:52:27Z)
• Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning [39.29964085305846]
  Methods that use pre-trained large language models directly as planners are currently impractical due to limited correctness of plans. In this work, we introduce a novel alternative paradigm that constructs an explicit world (domain) model in planning domain definition language (PDDL) and then uses it to plan with sound domain-independent planners.
  arXiv  Detail & Related papers  (2023-05-24T08:59:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.