Efficient Agentic Reasoning Through Self-Regulated Simulative Planning

• URL: http://arxiv.org/abs/2605.22138v1
• Date: Thu, 21 May 2026 08:11:54 GMT
• Title: Efficient Agentic Reasoning Through Self-Regulated Simulative Planning
• Abstract summary: We argue efficient agentic benefits from decision-making into three systems.<n>Self-regulation instantiates a principle we expect to extend beyond planning to how agents govern their own learning and adaptation.
• Score: 31.8332191338687
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: How should an agent decide when and how to plan? A dominant approach builds agents as reactive policies with adaptive computation (e.g., chain-of-thought), trained end-to-end expecting planning to emerge implicitly. Without control over the presence, structure, or horizon of planning, these systems dramatically increase reasoning length, yielding inefficient token use without reliable accuracy gains. We argue efficient agentic reasoning benefits from decomposing decision-making into three systems: simulative reasoning (System II) grounding deliberation in future-state prediction via a world model; self-regulation (System III) deciding when and how deeply to plan via a learned configurator; and reactive execution (System I) handling fine-grained action. Simulative reasoning provides unified planning across diverse tasks without per-domain engineering, while self-regulation ensures the planner is invoked only when needed. To test this, we develop SR$^2$AM (Self-Regulated Simulative Reasoning Agentic LLM), realizing both as distinct stages within an LLM's chain-of-thought, with the LLM as world model. We explore two instantiations: recording decisions from a prompted multi-module system (v0.1) and reconstructing structured plans from traces of pretrained reasoning LLMs (v1.0), trained via supervised then reinforcement learning (RL). Across math, science, tabular analysis, and web information seeking, v0.1-8B and v1.0-30B achieve Pass@1 competitive with 120-355B and 685B-1T parameter systems respectively, while v1.0-30B uses 25.8-95.3% fewer reasoning tokens than comparable agentic LLMs. RL increases average planning horizon by 22.8% while planning frequency grows only 2.0%, showing it learns to plan further ahead rather than more often. More broadly, learned self-regulation instantiates a principle we expect to extend beyond planning to how agents govern their own learning and adaptation.

Related papers

• Agentic Incident Response through Digital Twin-Enhanced Multiscale Planning [3.733830491643117]
  Incident response is currently managed by security operators using predefined playbooks.<n>Decision-theoretic approaches based on control, optimization, and reinforcement learning have been proposed to automate such planning tasks.<n>A promising approach is to use the security knowledge embedded in large language models to develop agentic response systems.
  arXiv  Detail & Related papers  (2026-08-03T16:03:16Z)
• Agentic LLM Planning via Step-Wise PDDL Simulation: An Empirical Characterisation [1.2832858109291982]
  Large language models (LLMs) can serve as viable planners alongside classical symbolic methods.<n>We present PyPDDLEngine, an open-source Planning Domain Definition Language (PDDL) simulation engine.<n>We evaluate four approaches on 102 International Planning Competition (IPC) Blocksworld instances under a uniform 180-second budget.
  arXiv  Detail & Related papers  (2026-03-06T09:16:49Z)
• rSIM: Incentivizing Reasoning Capabilities of LLMs via Reinforced Strategy Injection [49.74493901036598]
  Large language models (LLMs) are post-trained through reinforcement learning (RL) to evolve into Reasoning Language Models (RLMs)<n>This paper proposes a novel reinforced strategy injection mechanism (rSIM) that enables any LLM to become an RLM by employing a small planner.<n> Experimental results show that rSIM enables Qwen2.5-0.5B to become an RLM and significantly outperform Qwen2.5-14B.
  arXiv  Detail & Related papers  (2025-12-09T06:55:39Z)
• Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail [85.47497935739936]
  Alpamayo-R1 (AR1) is a vision-language-action model that integrates Chain of Causation reasoning with trajectory planning.<n>We show AR1 achieves 12% improvement in planning accuracy on challenging cases compared to a trajectory-only baseline.<n>We plan to release AR1 models and a subset of the CoC in a future update.
  arXiv  Detail & Related papers  (2025-10-30T01:25:34Z)
• Teaching LLMs to Plan: Logical Chain-of-Thought Instruction Tuning for Symbolic Planning [23.185497225384207]
  Large language models (LLMs) have demonstrated impressive capabilities across diverse tasks, yet their ability to perform structured symbolic planning remains limited.<n>We present a novel instruction tuning framework, PDDL-Instruct, designed to enhance LLMs' symbolic planning capabilities through logical chain-of-thought reasoning.
  arXiv  Detail & Related papers  (2025-09-14T02:42:34Z)
• Learning When to Plan: Efficiently Allocating Test-Time Compute for LLM Agents [35.79575378215309]
  Training large language models (LLMs) to reason via reinforcement learning (RL) significantly improves their problem-solving capabilities.<n>We introduce a conceptual framework formalizing dynamic planning for LLM agents, enabling them to flexibly decide when to allocate test-time compute for planning.<n>Experiments on the Crafter environment show that dynamic planning agents trained with this approach are more sample-efficient and consistently achieve more complex objectives.
  arXiv  Detail & Related papers  (2025-09-03T18:00:13Z)
• Planning in Strawberry Fields: Evaluating and Improving the Planning and Scheduling Capabilities of LRM o1 [15.863162558281616]
  We evaluate the planning capabilities of two LRMs (o1-preview and o1-mini) on both planning and scheduling benchmarks. We show that combining o1 models with external verifiers -- in a so-called LRM-Modulo system -- guarantees the correctness of the combined system's output.
  arXiv  Detail & Related papers  (2024-10-03T03:04:36Z)
• From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems [59.40480894948944]
  Large language model (LLM) empowered agents are able to solve decision-making problems in the physical world. Under this model, the LLM Planner navigates a partially observable Markov decision process (POMDP) by iteratively generating language-based subgoals via prompting. We prove that the pretrained LLM Planner effectively performs Bayesian aggregated imitation learning (BAIL) through in-context learning.
  arXiv  Detail & Related papers  (2024-05-30T09:42:54Z)
• LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoning [65.86754998249224]
  We develop a novel hybrid planner that leverages a conventional rule-based planner in conjunction with an LLM-based planner. Our approach navigates complex scenarios which existing planners struggle with, produces well-reasoned outputs while also remaining grounded through working alongside the rule-based approach.
  arXiv  Detail & Related papers  (2023-12-30T02:53:45Z)
• AdaPlanner: Adaptive Planning from Feedback with Language Models [56.367020818139665]
  Large language models (LLMs) have recently demonstrated the potential in acting as autonomous agents for sequential decision-making tasks. We propose a closed-loop approach, AdaPlanner, which allows the LLM agent to refine its self-generated plan adaptively in response to environmental feedback. To mitigate hallucination, we develop a code-style LLM prompt structure that facilitates plan generation across a variety of tasks, environments, and agent capabilities.
  arXiv  Detail & Related papers  (2023-05-26T05:52:27Z)
• Divide-and-Conquer Monte Carlo Tree Search For Goal-Directed Planning [78.65083326918351]
  We consider alternatives to an implicit sequential planning assumption. We propose Divide-and-Conquer Monte Carlo Tree Search (DC-MCTS) for approximating the optimal plan. We show that this algorithmic flexibility over planning order leads to improved results in navigation tasks in grid-worlds.
  arXiv  Detail & Related papers  (2020-04-23T18:08:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.