Self-Improvement for Fast, High-Quality Plan Generation

• URL: http://arxiv.org/abs/2605.03625v1
• Date: Tue, 05 May 2026 10:55:18 GMT
• Title: Self-Improvement for Fast, High-Quality Plan Generation
• Abstract summary: We show that a decoder-only transformer can generate high-quality plans for unseen problem instances.<n>We also show how to self-improve an initial model trained on sub-optimal data.
• Score: 13.377592170793994
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Generative models trained on synthetic plan data are a promising approach to generalized planning. Recent work has focused on finding any valid plan, rather than a high-quality solution. We address the challenge of producing high-quality plans, a computationally hard problem, in sub-exponential time. First, we demonstrate that, given optimal data, a decoder-only transformer can generate high-quality plans for unseen problem instances. Second, we show how to self-improve an initial model trained on sub-optimal data. Each round of self-improvement combines multiple model calls with graph search to generate improved plans, used for model fine-tuning. An experimental study on four domains: Blocksworld, Logistics, Labyrinth, and Sokoban, shows on average a 30% reduction in plan length over the source symbolic planner, with over 80% of plans being optimal, where the optimum is known. Plan quality is further improved by inference-time search. The model's latency scales sub-exponentially in contrast to the satisficing and optimal symbolic planners to which we compare. Together, these results suggest that self-improvement with generative models offers a scalable approach for high-quality plan generation.

Related papers

• PlanningBench: Generating Scalable and Verifiable Planning Data for Evaluating and Training Large Language Models [52.48858778580074]
  Planning is a fundamental capability for large language models (LLMs)<n>PlanningBench is a framework for generating scalable, diverse, verifiable planning data for both evaluation and training.
  arXiv  Detail & Related papers  (2026-05-20T08:10:15Z)
• PaT: Planning-after-Trial for Efficient Test-Time Code Generation [25.264318601226265]
  Planning-after-Trial (PaT) is an adaptive policy for code generation that invokes a planner only upon verification failure.<n>We propose PaT, an adaptive policy for code generation that invokes a planner only upon verification failure.
  arXiv  Detail & Related papers  (2026-05-08T05:09:18Z)
• Unified Vector Floorplan Generation via Markup Representation [36.228613615394856]
  We introduce Floorplan Markup Language (FML), a representation that encodes floorplan information within a single structured grammar.<n>We develop a transformer-based generative model, FMLM, capable of producing high-fidelity and functional floorplans under diverse conditions.
  arXiv  Detail & Related papers  (2026-04-06T17:04:30Z)
• Closing the Train-Test Gap in World Models for Gradient-Based Planning [64.36544881136405]
  We propose improved methods for training world models that enable efficient gradient-based planning.<n>At test time, our approach outperforms or matches the classical gradient-free cross-entropy method.
  arXiv  Detail & Related papers  (2025-12-10T18:59:45Z)
• Demo: Generative AI helps Radiotherapy Planning with User Preference [8.699769678493807]
  We introduce a novel generative model that predicts 3D dose distributions based solely on user-defined preference flavors.<n>These preferences enable planners to prioritize specific trade-offs between organs-at-risk (OARs) and planning target volumes (PTVs)
  arXiv  Detail & Related papers  (2025-12-08T16:49:21Z)
• Satisficing and Optimal Generalised Planning via Goal Regression (Extended Version) [16.43772461453855]
  Generalised planning (GP) refers to the task of synthesising programs that solve families of related planning problems.<n>We introduce a novel, yet simple method for GP: given a set of training problems, for each problem, compute an optimal plan for each goal atom in some order.<n>We formalise and prove the conditions under which our method is guaranteed to learn valid generalised plans and state space pruning axioms for search.
  arXiv  Detail & Related papers  (2025-11-14T09:16:32Z)
• DeepPlanner: Scaling Planning Capability for Deep Research Agents via Advantage Shaping [74.34061104176554]
  We propose DeepPlanner, an end-to-end RL framework that effectively enhances the planning capabilities of deep research agents.<n>Our approach shapes token-level advantage with an entropy-based term to allocate larger updates to high entropy tokens, and selectively upweights sample-level advantages for planning-intensive rollouts.
  arXiv  Detail & Related papers  (2025-10-14T20:47:05Z)
• World Modeling Makes a Better Planner: Dual Preference Optimization for Embodied Task Planning [60.100794160682646]
  We propose a new learning framework that jointly optimize state prediction and action selection through preference learning.<n>To automatically collect trajectories and stepwise preference data without human annotation, we introduce a tree search mechanism for extensive exploration via trial-and-error.<n>Our method significantly outperforms existing methods and GPT-4o when applied to Qwen2-VL (7B), LLaVA-1.6 (7B), and LLaMA-3.2 (11B)
  arXiv  Detail & Related papers  (2025-03-13T15:49:56Z)
• Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models [79.2162092822111]
  We systematically evaluate reinforcement learning (RL) and control-based methods on a suite of navigation tasks.<n>We employ a latent dynamics model using the Joint Embedding Predictive Architecture (JEPA) and employ it for planning.<n>Our results show that model-free RL benefits most from large amounts of high-quality data, whereas model-based planning generalizes better to unseen layouts.
  arXiv  Detail & Related papers  (2025-02-20T18:39:41Z)
• Experiment Planning with Function Approximation [49.50254688629728]
  We study the problem of experiment planning with function approximation in contextual bandit problems. We propose two experiment planning strategies compatible with function approximation. We show that a uniform sampler achieves competitive optimality rates in the setting where the number of actions is small.
  arXiv  Detail & Related papers  (2024-01-10T14:40:23Z)
• Planning as In-Painting: A Diffusion-Based Embodied Task Planning Framework for Environments under Uncertainty [56.30846158280031]
  Task planning for embodied AI has been one of the most challenging problems. We propose a task-agnostic method named 'planning as in-painting' The proposed framework achieves promising performances in various embodied AI tasks.
  arXiv  Detail & Related papers  (2023-12-02T10:07:17Z)
• Sequence-Based Plan Feasibility Prediction for Efficient Task and Motion Planning [36.300564378022315]
  We present a learning-enabled Task and Motion Planning (TAMP) algorithm for solving mobile manipulation problems in environments with many articulated and movable obstacles. The core of our algorithm is PIGINet, a novel Transformer-based learning method that takes in a task plan, the goal, and the initial state, and predicts the probability of finding motion trajectories associated with the task plan.
  arXiv  Detail & Related papers  (2022-11-03T04:12:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.