MemTrain: Self-Supervised Context Memory Training

• URL: http://arxiv.org/abs/2606.03197v1
• Date: Tue, 02 Jun 2026 05:56:24 GMT
• Title: MemTrain: Self-Supervised Context Memory Training
• Abstract summary: Existing memory-agent approaches are typically trained end-to-end with reinforcement learning on downstream tasks.<n>MemTrain is a self-supervised training framework for enhancing context-memory capability of LLM agents.<n>MemTrain consistently improves downstream memory-intensive reasoning performance across different models.
• Score: 40.64801276577851
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Memory is an indispensable capability for long-horizon LLM agents, enabling them to preserve and utilize information accumulated across extended interactions. Existing memory-agent approaches are typically trained end-to-end with reinforcement learning on downstream tasks. However, collecting high-quality annotated problems for memory-intensive scenarios is costly, and the resulting training data often lack sufficient diversity to cover general memory behaviors. In this work, we propose MemTrain, a self-supervised training framework for generally enhancing the context-memory capability of LLM agents for more effective downstream post-training. MemTrain introduces two coupled proxy tasks over unlabeled Wikipedia corpora: (1) an end-to-end masked reconstruction objective, which requires the model to recover masked entities after multiple rounds of memory updates, thereby encouraging memory maintenance from the final outcome perspective; and (2) an intermediate memory recall objective, which requires the model to reconstruct masked historical information using intermediate memory states, encouraging faithful compression and memory completeness throughout the interaction process. The two objectives are jointly optimized using GRPO. Extensive experiments on long-text QA and search-based QA benchmarks demonstrate that MemTrain consistently improves downstream memory-intensive reasoning performance across different models, achieving gains of up to 17.67 points over direct task-specific post-training.

Related papers

• Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents [72.3752981234916]
  Large Language Model (LLM) agents increasingly rely on external memory systems to accumulate experience across tasks.<n>We present Memory as a Controlled Process (MemCon), a framework that models memory operations as a Markov Decision Process.<n>MemCon consistently outperforms multiple memory baselines by up to 15.2 points in task success while reducing token consumption by 5-20%.
  arXiv  Detail & Related papers  (2026-07-15T08:32:40Z)
• AdMem: Advanced Memory for Task-solving Agents [13.611633561917513]
  Large Language Models (LLMs) show promise as tool-using agents but remain limited in long-horizon tasks that require remembering, organizing, and reusing knowledge.<n>We introduce a unified and automatic memory framework that integrates semantic, episodic, and procedural memory in a bi-level design combining short-term and long-term stores.
  arXiv  Detail & Related papers  (2026-06-05T00:11:57Z)
• Memory-R2: Fair Credit Assignment for Long-Horizon Memory-Augmented LLM Agents [27.2861945963127]
  Memory-augmented LLM agents enable interactions that extend beyond finite context windows.<n>Training such agents with reinforcement learning in multi-session environments is challenging.<n>We introduce Memory-R2, a training framework for memory-augmented LLM agents.
  arXiv  Detail & Related papers  (2026-05-20T22:02:00Z)
• MemSifter: Offloading LLM Memory Retrieval via Outcome-Driven Proxy Reasoning [78.46301394559903]
  Large Language Models (LLMs) are increasingly used for long-duration tasks.<n>Current methods face a trade-off between cost and accuracy.<n>MemSifter is a novel framework that offloads the memory retrieval process to a small-scale proxy model.
  arXiv  Detail & Related papers  (2026-03-03T02:57:38Z)
• Mem-T: Densifying Rewards for Long-Horizon Memory Agents [23.19373149519922]
  We introduce Mem-T, an autonomous memory agent that interfaces with a lightweight hierarchical memory database to perform dynamic updates and multi-turn retrieval over streaming inputs.<n>We also propose MoT-GRPO, a tree-guided reinforcement learning framework that transforms sparse terminal feedback into dense, step-wise supervision via memory operation tree backpropagation and hindsight credit assignment.
  arXiv  Detail & Related papers  (2026-01-30T14:23:33Z)
• Fine-Mem: Fine-Grained Feedback Alignment for Long-Horizon Memory Management [63.48041801851891]
  Fine-Mem is a unified framework designed for fine-grained feedback alignment.<n> Experiments on Memalpha and MemoryAgentBench demonstrate that Fine-Mem consistently outperforms strong baselines.
  arXiv  Detail & Related papers  (2026-01-13T11:06:17Z)
• Memo: Training Memory-Efficient Embodied Agents with Reinforcement Learning [53.72709564555407]
  Memo is a transformer-based architecture and training recipe for reinforcement learning.<n>It incorporates the creation and retrieval of memory by interleaving periodic summarization tokens with the inputs of a model during training.<n>We demonstrate Memo's effectiveness on a gridworld meta-RL benchmark and a multi-object navigation task in photo-realistic indoor settings.
  arXiv  Detail & Related papers  (2025-10-22T16:24:47Z)
• Mem-α: Learning Memory Construction via Reinforcement Learning [20.916677456417464]
  Large language model (LLM) agents are constrained by limited context windows.<n>Current memory-augmented agents depend on pre-defined instructions and tools for memory updates.<n>Mem-alpha is a reinforcement learning framework that trains agents to effectively manage complex memory systems.
  arXiv  Detail & Related papers  (2025-09-30T08:02:34Z)
• Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning [89.55738101744657]
  Large Language Models (LLMs) have demonstrated impressive capabilities across a wide range of NLP tasks, but they remain fundamentally stateless.<n>We present Memory-R1, a reinforcement learning framework that equips LLMs with the ability to actively manage and utilize external memory.
  arXiv  Detail & Related papers  (2025-08-27T12:26:55Z)
• MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents [84.62985963113245]
  We introduce MEM1, an end-to-end reinforcement learning framework that enables agents to operate with constant memory across long multi-turn tasks.<n>At each turn, MEM1 updates a compact shared internal state that jointly supports memory consolidation and reasoning.<n>We show that MEM1-7B improves performance by 3.5x while reducing memory usage by 3.7x compared to Qwen2.5-14B-Instruct on a 16-objective multi-hop QA task.
  arXiv  Detail & Related papers  (2025-06-18T19:44:46Z)
• CMT: A Memory Compression Method for Continual Knowledge Learning of Large Language Models [22.93893181000535]
  Large Language Models (LLMs) need to adapt to the continuous changes in data, tasks, and user preferences.<n>To address these challenges, this paper proposes the Compression Memory Training (CMT) method.<n>CMT compresses and extracts information from new documents to be stored in a memory bank.<n>When answering to queries related to these new documents, the model aggregates these document memories from the memory bank to better answer user questions.
  arXiv  Detail & Related papers  (2024-12-10T10:35:19Z)
• Think Before You Act: Decision Transformers with Working Memory [44.18926449252084]
  Decision Transformer-based decision-making agents have shown the ability to generalize across multiple tasks. We argue that this inefficiency stems from the forgetting phenomenon, in which a model memorizes its behaviors in parameters throughout training. We propose a working memory module to store, blend, and retrieve information for different downstream tasks.
  arXiv  Detail & Related papers  (2023-05-24T01:20:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.