InfoMem: Training Long-Context Memory Agents with Answer-Conditioned Information Gain

• URL: http://arxiv.org/abs/2606.03329v1
• Date: Tue, 02 Jun 2026 08:39:03 GMT
• Title: InfoMem: Training Long-Context Memory Agents with Answer-Conditioned Information Gain
• Abstract summary: Long-context tasks require LLMs to identify and preserve answer-relevant information from large contexts.<n>Existing RL-based chunk-wise agents either rely on sparse final-answer rewards or use lexical intermediate rewards for memory and retrieval actions.<n>We propose InfoMem, a reward mechanism for training chunk-wise memory agents that evaluates final-memory utility using answer-conditioned information.
• Score: 26.098768400948614
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-context tasks require LLMs to identify and preserve answer-relevant information from large contexts. Chunk-wise memory agents address this issue by sequentially reading document chunks, updating a compact memory, and generating the final answer from the accumulated memory. However, existing RL-based chunk-wise agents either rely on sparse final-answer rewards or use lexical intermediate rewards for memory and retrieval actions. These signals supervise task success or local overlap, but do not directly evaluate whether the final memory supports the ground-truth answer. We propose InfoMem, a reward mechanism for training chunk-wise memory agents that evaluates final-memory utility using answer-conditioned information. InfoMem measures how much the final memory increases the model's per-token log-likelihood of the ground-truth answer. To stabilize RL optimization, InfoMem applies this signal only to successful trajectories and normalizes it before reward composition. Under the same GRPO framework and training budget, InfoMem improves long-context memory-agent performance over comparable memory-agent RL baselines. Analyses show that effective final-memory rewards should operate on successful trajectories, be normalized before reward composition, and be conditioned on the answer rather than the query. Our code is available at https://github.com/GenSouKa1/InfoMem.

Related papers

• AWM: Answerable Working Memory for Long-Document VQA Agents [57.741922050360074]
  Working memory should carry answer-supporting evidence across page inspections for later grounded answering.<n>We introduce emphmemory-only answerability, a diagnostic that asks whether a reader can answer from the question and terminal working memory alone.<n>AWM-GRPO improves final-answer accuracy over the RAG baseline by 8.1 and 11.9 points on textscMMLongBench-Doc and textscLongDocURL and reduces the memory-missing-correct rate by 2.7 points over answer-only GRPO.
  arXiv  Detail & Related papers  (2026-08-26T10:35:25Z)
• TransMem: Transforming Hidden States into Memory for Large Language Models [29.656816378992414]
  Large language model (LLM) agents increasingly operate over long interaction histories.<n>TransMem transforms sparse historical hidden states from a frozen LLM backbone into reusable memory representations.
  arXiv  Detail & Related papers  (2026-07-31T05:11:04Z)
• AttriMem: Attribution-Guided Process Feedback for Agent Memory Learning [13.898789808338634]
  We propose an attribution-guided process-feedback framework for learning memory-construction policies with RL.<n>AttriMem augments the global outcome reward with local rewards derived from token-level contributions to the final answer.
  arXiv  Detail & Related papers  (2026-07-23T09:35:34Z)
• Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents [72.3752981234916]
  Large Language Model (LLM) agents increasingly rely on external memory systems to accumulate experience across tasks.<n>We present Memory as a Controlled Process (MemCon), a framework that models memory operations as a Markov Decision Process.<n>MemCon consistently outperforms multiple memory baselines by up to 15.2 points in task success while reducing token consumption by 5-20%.
  arXiv  Detail & Related papers  (2026-07-15T08:32:40Z)
• MEMPROBE: Probing Long-Term Agent Memory via Hidden User-State Recovery [58.64329475683699]
  We argue that long-term memory should be evaluated as an auditable post-interaction artifact.<n>We instantiate this view in MEMPROBE, a benchmark in which a memory-equipped agent assists simulated users.<n>We find that successful assistance and recoverable memory behave as distinct capabilities.
  arXiv  Detail & Related papers  (2026-06-23T13:52:46Z)
• MemTrain: Self-Supervised Context Memory Training [40.64801276577851]
  Existing memory-agent approaches are typically trained end-to-end with reinforcement learning on downstream tasks.<n>MemTrain is a self-supervised training framework for enhancing context-memory capability of LLM agents.<n>MemTrain consistently improves downstream memory-intensive reasoning performance across different models.
  arXiv  Detail & Related papers  (2026-06-02T05:56:24Z)
• ElasticMem: Latent Memory as a Learnable Resource for LLM Agents [25.70415765824153]
  ElasticMem is a memory-augmented LLM framework that learns to use memory as an elastic latent resource.<n>We evaluate ElasticMem on MemorySuite, covering memory-intensive QA and embodied agent control.
  arXiv  Detail & Related papers  (2026-05-29T00:34:40Z)
• MemGym: a Long-Horizon Memory Environment for LLM Agents [69.79226770543049]
  We present MemGym, a benchmark for agentic memory that unifies existing agent gyms and in-house memory-grounded pipelines.<n>MemGym reports memory-isolated scores that decouple memory performance from reasoning, retrieval, and tool-use ability.<n>Our synthetic pipelines for MEMGYM-CODEQA and MEMGYM-DR are length-controllable, ablation-verified at every stage, and tightly aligned with downstream scenarios.
  arXiv  Detail & Related papers  (2026-05-20T07:25:33Z)
• EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective [21.66000179149483]
  EvoMemBench is a unified benchmark organized along two axes: memory scope and memory content.<n>We compare 15 representative memory methods with strong long-context baselines under a standardized protocol.<n>Results show that current memory systems are still far from a general solution.
  arXiv  Detail & Related papers  (2026-05-18T13:54:38Z)
• MemRerank: Preference Memory for Personalized Product Reranking [14.982376048234386]
  We propose MemRerank, a preference memory framework that distills user purchase history into concise, query-independent signals for personalized product reranking.<n>Experiments with two LLM-based rerankers show that MemRerank consistently outperforms no-memory, raw-history, and off-the-shelf memory baselines.
  arXiv  Detail & Related papers  (2026-03-31T04:16:18Z)
• MemSifter: Offloading LLM Memory Retrieval via Outcome-Driven Proxy Reasoning [78.46301394559903]
  Large Language Models (LLMs) are increasingly used for long-duration tasks.<n>Current methods face a trade-off between cost and accuracy.<n>MemSifter is a novel framework that offloads the memory retrieval process to a small-scale proxy model.
  arXiv  Detail & Related papers  (2026-03-03T02:57:38Z)
• NextMem: Towards Latent Factual Memory for LLM-based Agents [58.35585202907478]
  NextMem is a latent factual memory framework that utilizes an autoregressive autoencoder to efficiently construct latent memory.<n>Extensive experiments demonstrate that NextMem achieves superior performance.
  arXiv  Detail & Related papers  (2026-02-26T14:35:27Z)
• MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks [55.145729491377374]
  Existing evaluations of agents with memory typically assess memorization and action in isolation.<n>We introduce MemoryArena, a unified evaluation gym for benchmarking agent memory in multi-session Memory-Agent-Environment loops.<n> MemoryArena supports evaluation across web navigation, preference-constrained planning, progressive information search, and sequential formal reasoning.
  arXiv  Detail & Related papers  (2026-02-18T09:49:14Z)
• Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents [57.38404718635204]
  Large language model (LLM) agents face fundamental limitations in long-horizon reasoning due to finite context windows.<n>Existing methods typically handle long-term memory (LTM) and short-term memory (STM) as separate components.<n>We propose Agentic Memory (AgeMem), a unified framework that integrates LTM and STM management directly into the agent's policy.
  arXiv  Detail & Related papers  (2026-01-05T08:24:16Z)
• From Single to Multi-Granularity: Toward Long-Term Memory Association and Selection of Conversational Agents [79.87304940020256]
  Large Language Models (LLMs) have been widely adopted in conversational agents.<n>MemGAS is a framework that enhances memory consolidation by constructing multi-granularity association, adaptive selection, and retrieval.<n> Experiments on four long-term memory benchmarks demonstrate that MemGAS outperforms state-of-the-art methods on both question answer and retrieval tasks.
  arXiv  Detail & Related papers  (2025-05-26T06:13:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.