Procedural Knowledge at Scale Improves Reasoning

• URL: http://arxiv.org/abs/2604.01348v1
• Date: Wed, 01 Apr 2026 20:01:47 GMT
• Title: Procedural Knowledge at Scale Improves Reasoning
• Abstract summary: We introduce Reasoning Memory, a framework for reasoning models that explicitly retrieves and reuses procedural knowledge at scale.<n> Reasoning Memory consistently outperforms RAG with document, trajectory, and template knowledge, as well as a compute-matched test-time scaling baseline.
• Score: 25.36077714467684
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Test-time scaling has emerged as an effective way to improve language models on challenging reasoning tasks. However, most existing methods treat each problem in isolation and do not systematically reuse knowledge from prior reasoning trajectories. In particular, they underutilize procedural knowledge: how to reframe a problem, choose an approach, and verify or backtrack when needed. We introduce Reasoning Memory, a retrieval-augmented generation (RAG) framework for reasoning models that explicitly retrieves and reuses procedural knowledge at scale. Starting from existing corpora of step-by-step reasoning trajectories, we decompose each trajectory into self-contained subquestion-subroutine pairs, yielding a datastore of 32 million compact procedural knowledge entries. At inference time, a lightweight in-thought prompt lets the model verbalize the core subquestion, retrieve relevant subroutines within its reasoning trace, and reason under diverse retrieved subroutines as implicit procedural priors. Across six math, science, and coding benchmarks, Reasoning Memory consistently outperforms RAG with document, trajectory, and template knowledge, as well as a compute-matched test-time scaling baseline. With a higher inference budget, it improves over no retrieval by up to 19.2% and over the strongest compute-matched baseline by 7.9% across task types. Ablation studies show that these gains come from two key factors: the broad procedural coverage of the source trajectories and our decomposition and retrieval design, which together enable effective extraction and reuse of procedural knowledge.

Related papers

• RING: Retrieval-Internalized Generation for Continual Large-Scale Knowledge Injection [88.56816599695391]
  Retrieval-augmented generation (RAG) improves factuality but adds latency and engineering overhead at serving time.<n>We propose RING (Retrieval-Internalized Generation), a holistic paradigm spanning both architecture and training.
  arXiv  Detail & Related papers  (2026-08-03T03:00:43Z)
• Trajectory-Informed Memory Generation for Self-Improving Agent Systems [4.933717407152962]
  LLM-powered agents face a persistent challenge: learning from their execution experiences to improve future performance.<n>We present a novel framework for automatically extracting actionable learnings from agent execution trajectories.<n>Our framework understands execution patterns, extracts structured learnings with provenance, and retrieves guidance tailored to specific task contexts.
  arXiv  Detail & Related papers  (2026-03-11T09:54:09Z)
• Multi-hop Reasoning via Early Knowledge Alignment [68.28168992785896]
  Early Knowledge Alignment (EKA) aims to align Large Language Models with contextually relevant retrieved knowledge.<n>EKA significantly improves retrieval precision, reduces cascading errors, and enhances both performance and efficiency.<n>EKA proves effective as a versatile, training-free inference strategy that scales seamlessly to large models.
  arXiv  Detail & Related papers  (2025-12-23T08:14:44Z)
• A Benchmark for Procedural Memory Retrieval in Language Agents [0.023227405857540805]
  Current AI agents excel in familiar settings, but fail sharply when faced with novel tasks with unseenProc.<n>We present the first benchmark that isolates procedural memory retrieval from task execution.<n>Our results expose a clear generalization cliff: embedding-based methods perform strongly on familiar contexts, yet degrade considerably on novel ones.
  arXiv  Detail & Related papers  (2025-11-21T08:08:53Z)
• Adaptive Test-Time Reasoning via Reward-Guided Dual-Phase Search [62.1546099504045]
  We propose a dual-phase test-time scaling framework that separates reasoning into planning and execution.<n>Specifically, we decompose reasoning trajectories and develop reward models for each phase, enabling the search to explore and prune plans and executions separately.<n> Experiments on both mathematical reasoning and code generation benchmarks demonstrate that our approach consistently improves accuracy while reducing computation redundant.
  arXiv  Detail & Related papers  (2025-09-29T19:27:23Z)
• Retrieval Feedback Memory Enhancement Large Model Retrieval Generation Method [10.654167904580838]
  Large Language Models (LLMs) face inherent limitations such as constrained parametric knowledge and high retraining costs.<n>We propose Retrieval Feedback and Memory Retrieval Augmented Generation (RFM-RAG)<n>RFM-RAG transforms the stateless retrieval of previous methods into stateful continuous knowledge management by constructing a dynamic evidence pool.
  arXiv  Detail & Related papers  (2025-08-25T10:13:02Z)
• Beyond Memorization: Extending Reasoning Depth with Recurrence, Memory and Test-Time Compute Scaling [60.63703438729223]
  We show how different architectures and training methods affect model multi-step reasoning capabilities.<n>We confirm that increasing model depth plays a crucial role for sequential computations.
  arXiv  Detail & Related papers  (2025-08-22T18:57:08Z)
• FrugalRAG: Learning to retrieve and reason for multi-hop QA [10.193015391271535]
  Large-scale fine-tuning is not needed to improve RAG metrics.<n>Supervised and RL-based fine-tuning can help RAG from the perspective of frugality.
  arXiv  Detail & Related papers  (2025-07-10T11:02:13Z)
• Log-Augmented Generation: Scaling Test-Time Reasoning with Reusable Computation [80.69067017594709]
  Large language models (LLMs) and their agentic counterparts struggle to retain reasoning from previous tasks.<n>We propose a novel framework, log-augmented generation (LAG) that directly reuses prior computation and reasoning from past logs at test time.<n>Our method significantly outperforms standard agentic systems that do not utilize logs.
  arXiv  Detail & Related papers  (2025-05-20T14:14:38Z)
• Disentangling Memory and Reasoning Ability in Large Language Models [97.26827060106581]
  We propose a new inference paradigm that decomposes the complex inference process into two distinct and clear actions.<n>Our experiment results show that this decomposition improves model performance and enhances the interpretability of the inference process.
  arXiv  Detail & Related papers  (2024-11-20T17:55:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.