Latent Planning Emerges with Scale

• URL: http://arxiv.org/abs/2604.12493v1
• Date: Tue, 14 Apr 2026 09:18:53 GMT
• Title: Latent Planning Emerges with Scale
• Abstract summary: We study the Qwen-3 family (0.6B-14B) on simple planning tasks, finding that latent planning ability increases with scale.<n>On the more complex task of completing rhyming couplets, we find that models often identify a rhyme ahead of time, but even large models seldom plan far ahead.
• Score: 2.8074191213147652
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLMs can perform seemingly planning-intensive tasks, like writing coherent stories or functioning code, without explicitly verbalizing a plan; however, the extent to which they implicitly plan is unknown. In this paper, we define latent planning as occurring when LLMs possess internal planning representations that (1) cause the generation of a specific future token or concept, and (2) shape preceding context to license said future token or concept. We study the Qwen-3 family (0.6B-14B) on simple planning tasks, finding that latent planning ability increases with scale. Models that plan possess features that represent a planned-for word like "accountant", and cause them to output "an" rather than "a"; moreover, even the less-successful Qwen-3 4B-8B have nascent planning mechanisms. On the more complex task of completing rhyming couplets, we find that models often identify a rhyme ahead of time, but even large models seldom plan far ahead. However, we can elicit some planning that increases with scale when steering models towards planned words in prose. In sum, we offer a framework for measuring planning and mechanistic evidence of how models' planning abilities grow with scale.

Related papers

• PlanningBench: Generating Scalable and Verifiable Planning Data for Evaluating and Training Large Language Models [52.48858778580074]
  Planning is a fundamental capability for large language models (LLMs)<n>PlanningBench is a framework for generating scalable, diverse, verifiable planning data for both evaluation and training.
  arXiv  Detail & Related papers  (2026-05-20T08:10:15Z)
• Enhancing Cognitive Robotics with Commonsense through LLM-Generated Preconditions and Subgoals [0.0]
  Traditional symbolic planners need hidden preconditions and small subgoals to be written explicitly.<n>In this project we combine a Large Language Model with symbolic planning.<n>Our system produces more valid plans, achieves a higher task success rate, and adapts better when the environment changes.
  arXiv  Detail & Related papers  (2025-11-24T11:16:41Z)
• Detecting and Characterizing Planning in Language Models [1.320426480090921]
  We present formal and causally grounded criteria for detecting planning and operationalize them as a semi-automated annotation pipeline.<n>We apply this pipeline to both base and instruction-tuned Gemma-2-2B models on the MBPP code generation benchmark and a poem generation task.<n>Our findings show that planning is not universal: unlike Haiku, Gemma-2-2B solves the same poem generation task through improvisation, and on MBPP it switches between planning and improvisation across similar tasks and even successive token predictions.
  arXiv  Detail & Related papers  (2025-08-25T14:59:46Z)
• Wide-Horizon Thinking and Simulation-Based Evaluation for Real-World LLM Planning with Multifaceted Constraints [39.01715254437105]
  This paper introduces the Multiple Aspects of Planning (MAoP) to solve planning problems with multifaceted constraints.<n>Instead of direct planning, MAoP leverages the strategist to conduct pre-planning from various aspects and provide the planning blueprint for planners.
  arXiv  Detail & Related papers  (2025-06-14T09:37:59Z)
• Counting and Reasoning with Plans [19.223883847258158]
  We present the first study to quantitative and qualitative reasoning on the plan space.<n>On the theoretical side, we study its complexity, which gives rise to rich reasoning modes.<n>Thereby, we transform a planning task into a propositional formula and use knowledge compilation to count different plans.
  arXiv  Detail & Related papers  (2025-01-31T20:03:51Z)
• Hindsight Planner: A Closed-Loop Few-Shot Planner for Embodied Instruction Following [62.10809033451526]
  This work focuses on building a task planner for Embodied Instruction Following (EIF) using Large Language Models (LLMs)<n>We frame the task as a Partially Observable Markov Decision Process (POMDP) and aim to develop a robust planner under a few-shot assumption.<n>Our experiments on the ALFRED dataset indicate that our planner achieves competitive performance under a few-shot assumption.
  arXiv  Detail & Related papers  (2024-12-27T10:05:45Z)
• LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoning [65.86754998249224]
  We develop a novel hybrid planner that leverages a conventional rule-based planner in conjunction with an LLM-based planner. Our approach navigates complex scenarios which existing planners struggle with, produces well-reasoned outputs while also remaining grounded through working alongside the rule-based approach.
  arXiv  Detail & Related papers  (2023-12-30T02:53:45Z)
• Planning as In-Painting: A Diffusion-Based Embodied Task Planning Framework for Environments under Uncertainty [56.30846158280031]
  Task planning for embodied AI has been one of the most challenging problems. We propose a task-agnostic method named 'planning as in-painting' The proposed framework achieves promising performances in various embodied AI tasks.
  arXiv  Detail & Related papers  (2023-12-02T10:07:17Z)
• PlaSma: Making Small Language Models Better Procedural Knowledge Models for (Counterfactual) Planning [77.03847056008598]
  PlaSma is a novel two-pronged approach to endow small language models with procedural knowledge and (constrained) language planning capabilities. We develop symbolic procedural knowledge distillation to enhance the commonsense knowledge in small language models and an inference-time algorithm to facilitate more structured and accurate reasoning.
  arXiv  Detail & Related papers  (2023-05-31T00:55:40Z)
• AdaPlanner: Adaptive Planning from Feedback with Language Models [56.367020818139665]
  Large language models (LLMs) have recently demonstrated the potential in acting as autonomous agents for sequential decision-making tasks. We propose a closed-loop approach, AdaPlanner, which allows the LLM agent to refine its self-generated plan adaptively in response to environmental feedback. To mitigate hallucination, we develop a code-style LLM prompt structure that facilitates plan generation across a variety of tasks, environments, and agent capabilities.
  arXiv  Detail & Related papers  (2023-05-26T05:52:27Z)
• PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change [34.93870615625937]
  PlanBench is a benchmark suite based on the kinds of domains used in the automated planning community. PlanBench provides sufficient diversity in both the task domains and the specific planning capabilities.
  arXiv  Detail & Related papers  (2022-06-21T16:15:27Z)
• Planning with Learned Object Importance in Large Problem Instances using Graph Neural Networks [28.488201307961624]
  Real-world planning problems often involve hundreds or even thousands of objects. We propose a graph neural network architecture for predicting object importance in a single inference pass. Our approach treats the planner and transition model as black boxes, and can be used with any off-the-shelf planner.
  arXiv  Detail & Related papers  (2020-09-11T18:55:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.