MILES: Modular Instruction Memory with Learnable Selection for Self-Improving LLM Reasoning

• URL: http://arxiv.org/abs/2607.06974v1
• Date: Wed, 08 Jul 2026 03:51:37 GMT
• Title: MILES: Modular Instruction Memory with Learnable Selection for Self-Improving LLM Reasoning
• Abstract summary: MILES is a framework that dynamically expands step-wise memory and applies correctness-optimized memory composition under realistic test-time constraints.<n> MILES consistently matches or outperforms prior methods while achieving superior accuracy-efficiency tradeoffs.
• Score: 17.668597279485407
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) increasingly improve their reasoning at test time via additional computation, yet most existing works treat each problem in isolation. When problems arrive sequentially, accumulating reusable experience across them can further improve performance. Existing memory-based methods either store whole-solution templates that generalize poorly to novel problems or use heuristic step-level selection that is not optimized for final-answer correctness. Learning selection policies requires large-scale training data and fixed action spaces, making such approaches unsuitable for test-time settings where memory expands incrementally and only limited supervision is available. We propose MILES (Modular Instruction Memory with LEarnable Selection for self-improving LLM reasoning), a framework that dynamically expands step-wise memory and applies correctness-optimized memory composition under realistic test-time constraints. MILES maintains modular memory units consisting of asymmetric pairs of sub-goal embeddings and sub-instructions, each associated with a learnable selection head. This memory structure enables a coarse-to-fine retrieval mechanism: The coarse level enables memory expansion and collects supervision for training selection heads from confident samples, while the fine stage applies learned selection heads to rerank coarse-level candidates and guide reasoning for uncertain samples. MILES consistently matches or outperforms prior methods while achieving superior accuracy-efficiency tradeoffs. Extensive experiments demonstrate its effectiveness, robustness, and transferability.

Related papers

• Verifiable Memory: Learning Unified Memory Management with Local and Global Verifiers for Large Language Model Agents [12.763061404384601]
  Large language model (LLM) agents must retain reusable information, control a bounded active context, and recover earlier evidence during long-horizon interaction.<n>We present VerMem, a framework that represents LTM, active context, and episodic history as distinct states and controls them with one memory operation policy.
  arXiv  Detail & Related papers  (2026-08-04T05:06:24Z)
• Unifying Data, Memory, and Compute Efficiency in LLM training: A Survey [14.047214709513135]
  This survey adopts a constraint-centric perspective and organizes recent progress around three coupled bottlenecks: data efficiency (what to train on), memory efficiency (how to fit training), and compute budget awareness (when and where to spend FLOPs)<n>We highlight emerging evidence that different notions of good data dominate in different regimes, implying that optimal subsets depend on the task objective and resource budget rather than being universal.<n>We frame training and inference as compute-governed processes in which optimization, data selection, and decoding must explicitly account for finite FLOP budgets.
  arXiv  Detail & Related papers  (2026-06-09T11:09:58Z)
• Learning What to Remember: Observability-Safe Memory Retention via Constrained Optimization for Long-Horizon Language Agents [16.193335398478386]
  Long-horizon language agents accumulate observations, reasoning traces, and retrieved facts windows.<n>Existing systems treat retention as local and do not model long-term consequences under observability constraints.<n>We propose OSL-MR, a learning-augmented framework that enforces a strict separation between online-observable features and offline-available supervision.
  arXiv  Detail & Related papers  (2026-06-09T09:15:33Z)
• MIITA: Memory-Induced Inference-Time Adaptation for Continual Learning with Small Language Models [64.2820121997882]
  MIITA is a Memory-Induced Inference-Time Adaptation framework for supervised CL under constrained storage.<n>It stores supervised experiences as compact correction-direction prototypes with semantic anchors, and retrieves them at inference time using semantic and uncertainty-based cues.<n>Experiments show that MIITA consistently improves final performance and mitigates forgetting under fixed memory budgets.
  arXiv  Detail & Related papers  (2026-05-20T03:03:04Z)
• GoTTA be Diverse: Rethinking Memory Policies for Test-Time Adaptation [54.70334474895361]
  Test-time adaptation (TTA) enables a pre-trained model to adapt online to an unlabeled test stream under distribution shift.<n>While most TTA research focuses on the adaptation objective, practical streams also depend critically on the memory used to select which test samples drive adaptation.<n>We introduce Guided Observational Test-Time Adaptation (GOTTA), a family of diversity-aware memory policies that combine class-balanced allocation with feature-space diversity.
  arXiv  Detail & Related papers  (2026-05-19T14:18:46Z)
• Is One Score Enough? Rethinking the Evaluation of Sequentially Evolving LLM Memory [50.857546269660276]
  We introduce SeqMem-Eval, a diagnostic evaluation framework for sequentially evolving large language models (LLMs) memory.<n>Rather than focusing only on final performance, SeqMem-Eval evaluates how memory states evolve, generalize, consolidate experience, and retain useful information during sequential inference.
  arXiv  Detail & Related papers  (2026-05-14T20:15:22Z)
• NextMem: Towards Latent Factual Memory for LLM-based Agents [58.35585202907478]
  NextMem is a latent factual memory framework that utilizes an autoregressive autoencoder to efficiently construct latent memory.<n>Extensive experiments demonstrate that NextMem achieves superior performance.
  arXiv  Detail & Related papers  (2026-02-26T14:35:27Z)
• Fine-Mem: Fine-Grained Feedback Alignment for Long-Horizon Memory Management [63.48041801851891]
  Fine-Mem is a unified framework designed for fine-grained feedback alignment.<n> Experiments on Memalpha and MemoryAgentBench demonstrate that Fine-Mem consistently outperforms strong baselines.
  arXiv  Detail & Related papers  (2026-01-13T11:06:17Z)
• Learn to Memorize: Optimizing LLM-based Agents with Adaptive Memory Framework [33.739298910759544]
  We propose to optimize LLM-based agents with an adaptive and data-driven memory framework by modeling memory cycles.<n>Specifically, we design an MoE gate function to facilitate memory retrieval, propose a learnable aggregation process to improve memory utilization, and develop task-specific reflection to adapt memory storage.
  arXiv  Detail & Related papers  (2025-08-15T12:22:52Z)
• PARMESAN: Parameter-Free Memory Search and Transduction for Dense Prediction Tasks [5.5127111704068374]
  This work addresses flexibility in deep learning by means of transductive reasoning.<n>We propose PARMESAN, a scalable method which leverages a memory module for solving dense prediction tasks.<n>Our method is compatible with commonly used architectures and canonically transfers to 1D, 2D, and 3D grid-based data.
  arXiv  Detail & Related papers  (2024-03-18T12:55:40Z)
• Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark [166.40879020706151]
  This paper proposes a shift towards BP-free, zeroth-order (ZO) optimization as a solution for reducing memory costs during fine-tuning. Unlike traditional ZO-SGD methods, our work expands the exploration to a wider array of ZO optimization techniques. Our study unveils previously overlooked optimization principles, highlighting the importance of task alignment, the role of the forward gradient method, and the balance between algorithm complexity and fine-tuning performance.
  arXiv  Detail & Related papers  (2024-02-18T14:08:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.