Unlocking the Working Memory of Large Language Models for Latent Reasoning

• URL: http://arxiv.org/abs/2605.30343v1
• Date: Thu, 28 May 2026 17:59:49 GMT
• Title: Unlocking the Working Memory of Large Language Models for Latent Reasoning
• Abstract summary: Reasoning in Memory (RiM) is a latent reasoning method that replaces the autoregressive generation of reasoning steps with memory blocks.<n>RiM matches or exceeds existing latent reasoning methods while avoiding the autoregressive generation of thoughts.
• Score: 21.924207996049407
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: To improve the reasoning capabilities of large language models, test-time compute is typically scaled by generating intermediate tokens before the final answer. However, this couples reasoning to autoregressive generation and thereby conflates internal computation with external communication. In contrast, human cognition can use working memory to hold and manipulate information internally without the need to externalize intermediate thoughts. Drawing on this principle, we introduce Reasoning in Memory (RiM), a latent reasoning method that replaces the autoregressive generation of reasoning steps with memory blocks. These memory blocks are fixed sequences of special tokens that unlock the working-memory capacity of large language models. Since they are fixed rather than generated, they can be processed in a single forward pass, enabling compute-efficient latent reasoning. To operationalize these memory blocks, we employ a two-stage curriculum. First, we ground them by predicting explicit reasoning steps after each memory block. Second, we discard this step-level supervision and iteratively refine the final answer after each memory block. Our experiments on reasoning benchmarks show that, across language models of different families and sizes, RiM matches or exceeds existing latent reasoning methods while avoiding the autoregressive generation of thoughts. These results demonstrate that large language models can be trained to use working memory as an effective mechanism for latent reasoning.

Related papers

• MemGuard: Preventing Memory Contamination in Long-Term Memory-Augmented Large Language Models [56.31411457917676]
  We introduce MemGuard, a type-aware memory framework that preserves functional memory boundaries during memory construction and retrieval.<n>Across hallucination and long-horizon conversation benchmarks, MemGuard improves memory reliability by up to 28.27% while retrieving up to 5.8x fewer memory tokens than prior methods.
  arXiv  Detail & Related papers  (2026-05-27T06:04:19Z)
• Simulating Human Memory with Language Models [15.781024764769024]
  We show that out-of-the-box language models exhibit better memory than humans, even when prompted to imitate human behavior.<n>We then show that better prompting strategies and the use of a compactor can cause language models to forget content in a more human-like way.
  arXiv  Detail & Related papers  (2026-05-25T10:39:08Z)
• Evaluating Long-Term Memory for Long-Context Question Answering [100.1267054069757]
  We present a systematic evaluation of memory-augmented methods using LoCoMo, a benchmark of synthetic long-context dialogues annotated for question-answering tasks.<n>Our findings show that memory-augmented approaches reduce token usage by over 90% while maintaining competitive accuracy.
  arXiv  Detail & Related papers  (2025-10-27T18:03:50Z)
• Pre-Storage Reasoning for Episodic Memory: Shifting Inference Burden to Memory for Personalized Dialogue [13.558061425427688]
  PREMem is a novel approach that shifts complex reasoning processes from inference to memory construction.<n>It creates enriched representations while reducing computational demands during interactions.<n> Experiments show significant performance improvements across all model sizes.
  arXiv  Detail & Related papers  (2025-09-13T15:18:08Z)
• Human-like fleeting memory improves language learning but impairs reading time prediction in transformer language models [0.0]
  We investigate the hypothesized benefit of fleeting memory for language learning in experiments on transformer language models.<n>We find that fleeting memory consistently improves language learning but, unexpectedly, impairs surprisal-based prediction of human reading times.
  arXiv  Detail & Related papers  (2025-08-07T19:17:53Z)
• HMT: Hierarchical Memory Transformer for Efficient Long Context Language Processing [33.720656946186885]
  Hierarchical Memory Transformer (HMT) is a novel framework that facilitates a model's long-context processing ability.<n>HMT consistently improves the long-context processing ability of existing models.
  arXiv  Detail & Related papers  (2024-05-09T19:32:49Z)
• MoT: Memory-of-Thought Enables ChatGPT to Self-Improve [73.90376920653507]
  We propose a framework, Memory-of-Thought, to let the Large Language Models self-improve without annotated datasets and parameter updates. Experimental results show that MoT can help ChatGPT significantly improve its abilities in arithmetic reasoning, commonsense reasoning, factual reasoning, and natural language inference.
  arXiv  Detail & Related papers  (2023-05-09T05:25:05Z)
• Lift Yourself Up: Retrieval-augmented Text Generation with Self Memory [72.36736686941671]
  We propose a novel framework, selfmem, for improving retrieval-augmented generation models. Selfmem iteratively employs a retrieval-augmented generator to create an unbounded memory pool and using a memory selector to choose one output as memory for the subsequent generation round. We evaluate the effectiveness of selfmem on three distinct text generation tasks.
  arXiv  Detail & Related papers  (2023-05-03T21:40:54Z)
• Learning to Reason and Memorize with Self-Notes [51.17609489687686]
  Large language models have been shown to struggle with multi-step reasoning. We propose a simple method for solving both of these problems by allowing the model to take Self-Notes.
  arXiv  Detail & Related papers  (2023-05-01T14:02:48Z)
• LaMemo: Language Modeling with Look-Ahead Memory [50.6248714811912]
  We propose Look-Ahead Memory (LaMemo) that enhances the recurrence memory by incrementally attending to the right-side tokens. LaMemo embraces bi-directional attention and segment recurrence with an additional overhead only linearly proportional to the memory length. Experiments on widely used language modeling benchmarks demonstrate its superiority over the baselines equipped with different types of memory.
  arXiv  Detail & Related papers  (2022-04-15T06:11:25Z)
• Kanerva++: extending The Kanerva Machine with differentiable, locally block allocated latent memory [75.65949969000596]
  Episodic and semantic memory are critical components of the human memory model. We develop a new principled Bayesian memory allocation scheme that bridges the gap between episodic and semantic memory. We demonstrate that this allocation scheme improves performance in memory conditional image generation.
  arXiv  Detail & Related papers  (2021-02-20T18:40:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.