AutoPlanBench: Automatically generating benchmarks for LLM planners from PDDL

• URL: http://arxiv.org/abs/2311.09830v2
• Date: Fri, 9 Feb 2024 09:48:41 GMT
• Title: AutoPlanBench: Automatically generating benchmarks for LLM planners from PDDL
• Authors: Katharina Stein, Daniel Fi\v{s}er, J\"org Hoffmann and Alexander Koller
• Abstract summary: We present AutoPlanBench, a novel method for automatically converting planning benchmarks written in PDDL into textual descriptions. We show that while the best LLM planners do well on some planning tasks, others remain out of reach of current methods.
• Score: 52.005042190810116
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: LLMs are being increasingly used for planning-style tasks, but their capabilities for planning and reasoning are poorly understood. We present AutoPlanBench, a novel method for automatically converting planning benchmarks written in PDDL into textual descriptions and offer a benchmark dataset created with our method. We show that while the best LLM planners do well on some planning tasks, others remain out of reach of current methods.

Related papers

Err

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.