DeferMem: Query-Time Evidence Distillation via Reinforcement Learning for Long-Term Memory QA

• URL: http://arxiv.org/abs/2605.22411v1
• Date: Thu, 21 May 2026 12:36:46 GMT
• Title: DeferMem: Query-Time Evidence Distillation via Reinforcement Learning for Long-Term Memory QA
• Abstract summary: Large language model (LLM) agents still struggle with long-term memory question answering.<n>We present DeferMem, a long-term memory framework that decouples this problem into high-recall candidate retrieval and query-conditioned evidence distillation.<n>On LoCoMo and LongMemEval-S, DeferMem surpasses strong baselines in QA accuracy and memory-system efficiency.
• Score: 3.56156695760535
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language model (LLM) agents still struggle with long-term memory question answering, where answer-supporting evidence is often scattered across long conversational histories and buried in substantial irrelevant content. Existing memory systems typically process memory before future queries are known, then retrieve the resulting units based on similarity rather than their utility for answering the query. This workflow leaves downstream answerers to denoise retrieved candidates and reconstruct query-specific evidence. We present DeferMem, a long-term memory framework that decouples this problem into high-recall candidate retrieval and query-conditioned evidence distillation. DeferMem uses a lightweight segment-link structure to organize raw history and retrieve broad candidates at query time. It then applies a memory distiller trained with DistillPO, our reinforcement learning algorithm for distilling the high-recall but highly noisy candidates into a set of faithful, self-contained, and query-conditioned evidence. DistillPO formulates post-retrieval evidence distillation as a structured action comprising message selection and evidence rewriting. It optimizes this action with a decomposed-and-gated reward pipeline and structure-aligned advantage assignment, gating reward components from validity to quality checks while exposing task-level correctness feedback early and assigning each reward to its responsible output span. On LoCoMo and LongMemEval-S, DeferMem surpasses strong baselines in QA accuracy and memory-system efficiency, achieving the highest QA accuracy with the fastest runtime and zero commercial-API token cost for memory operations.

Related papers

• CueMem: Cue-Guided Context Reconstruction for Long-Term Conversational Memory [38.76918926805768]
  CueMem is a cue-guided framework that treats extracted memory records as retrieval cues.<n>It reconstructs query-relevant dialogue context from their source turns.<n>Experiments on LoCoMo and LongMemEval show that CueMem consistently outperforms representative long-term memory baselines.
  arXiv  Detail & Related papers  (2026-09-11T02:27:22Z)
• GROVE: Growing and Reasoning over Temporally Stratified Memory from Streaming Video Experience [67.55136435794206]
  A wearable assistant should both answer questions about its visual history and recognize when that history is useful to the present situation.<n>We introduce GROVE, a training-free framework that supports both behaviors with one memory grown causally from a continuous video stream.<n>GROVE retains fine-grained perceptual evidence and incrementally consolidates it into time-stamped moments, coherent episodes, and recurring cross-day patterns.
  arXiv  Detail & Related papers  (2026-08-03T15:35:28Z)
• Stop When Memory Suffices: Evidence-Conditioned Progressive Execution for LLM Agents [5.4775608854724345]
  Existing memory systems either compress and structure histories for efficient access or perform deep research over broader trajectories.<n>This raises a key question: can a memory system achieve strong answer quality while maintaining low online latency?<n>We introduce Router-Mem, an evidence-conditioned progressive execution framework for long-horizon agent memory.
  arXiv  Detail & Related papers  (2026-08-02T14:49:24Z)
• PMMC: Prospective Multimodal Memory Compilation for Long-Term LVLM Agents [19.297991318803508]
  Long-term memory is essential for LVLM agents to maintain consistency and integrate information across extended multimodal interactions.<n>We propose Prospective Multimodal Memory Compilation, a framework that shifts part of the memory reasoning process from query time to memory consolidation time.<n> Experiments on multimodal long-term memory benchmarks show that our method improves answer quality and visual evidence recall while reducing query-time token and latency costs.
  arXiv  Detail & Related papers  (2026-08-02T03:20:54Z)
• MemChain: Learning Interpretable Memory Traces for Memory-Augmented LLM Agents [26.87993979719045]
  Memory-augmented LLM agents typically answer queries by retrieving relevant memories and feeding them directly to an answer model.<n>We propose MemChain, a trainable post-retrieval memory policy that transforms retrieved candidates into answer-facing active memory.<n>We show that MemChain consistently achieves state-of-the-art performance across both closed-source and open-weight frozen answer models.
  arXiv  Detail & Related papers  (2026-07-27T07:37:43Z)
• MemOps: Benchmarking Lifecycle Memory Operations in Long-Horizon Conversations [50.24315431387575]
  We introduce MemOps, a benchmark that reformulates conversational memory as a sequence of lifecycle operations.<n>MemOps represents each memory event with a structured trace specifying its trigger, target, scope, state transition, and supporting evidence.<n>Across long-context, retrieval-based, parametric and managed-memory systems, MemOps disentangles failure modes that final-answer accuracy alone conceals.
  arXiv  Detail & Related papers  (2026-07-14T15:33:44Z)
• AdaMem: Adaptive User-Centric Memory for Long-Horizon Dialogue Agents [49.63422082885992]
  We propose AdaMem, an adaptive user-centric memory framework for long-horizon dialogue agents.<n>AdaMem organizes dialogue history into working, episodic, persona, and graph memories.<n>We evaluate AdaMem on the LoCoMo and PERSONAMEM benchmarks for long-horizon reasoning and user modeling.
  arXiv  Detail & Related papers  (2026-03-17T13:22:54Z)
• Learning to Remember: End-to-End Training of Memory Agents for Long-Context Reasoning [18.621823772319154]
  We propose an end-to-end reinforcement learning framework that unifies memory operations and question answering within a single policy.<n>UMA maintains a dual memory representation: a compact core summary for global context and a structured Memory Bank that supports explicit CRUD.<n>Across 13 datasets spanning Ledger-QA, Test-Time Learning, and Accurate Retrieval, UMA substantially outperforms long-context and RAG baselines on dynamic reasoning and learning tasks.
  arXiv  Detail & Related papers  (2026-02-13T16:54:23Z)
• AMA: Adaptive Memory via Multi-Agent Collaboration [54.490349689939166]
  We propose Adaptive Memory via Multi-Agent Collaboration (AMA), a novel framework that leverages coordinated agents to manage memory across multiple granularities.<n>AMA significantly outperforms state-of-the-art baselines while reducing token consumption by approximately 80% compared to full-context methods.
  arXiv  Detail & Related papers  (2026-01-28T08:09:49Z)
• Look Back to Reason Forward: Revisitable Memory for Long-Context LLM Agents [33.617262543252494]
  We present ReMemR1, a memory-augmented agent with callback-enhanced memory that allows selective retrieval from the entire memory history.<n>We also propose Reinforcement Learning with Multi-Level Rewards (RLMLR), which combines final-answer rewards with dense, step-level signals that guide effective memory use.
  arXiv  Detail & Related papers  (2025-09-27T01:36:46Z)
• From Single to Multi-Granularity: Toward Long-Term Memory Association and Selection of Conversational Agents [79.87304940020256]
  Large Language Models (LLMs) have been widely adopted in conversational agents.<n>MemGAS is a framework that enhances memory consolidation by constructing multi-granularity association, adaptive selection, and retrieval.<n> Experiments on four long-term memory benchmarks demonstrate that MemGAS outperforms state-of-the-art methods on both question answer and retrieval tasks.
  arXiv  Detail & Related papers  (2025-05-26T06:13:07Z)
• LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory [68.97819665784442]
  We introduce LongMemEval, a benchmark designed to evaluate five core long-term memory abilities of chat assistants.<n>LongMemEval presents a significant challenge to existing long-term memory systems.<n>We present a unified framework that breaks down the long-term memory design into three stages: indexing, retrieval, and reading.
  arXiv  Detail & Related papers  (2024-10-14T17:59:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.