AttriMem: Attribution-Guided Process Feedback for Agent Memory Learning

• URL: http://arxiv.org/abs/2607.21106v1
• Date: Thu, 23 Jul 2026 09:35:34 GMT
• Title: AttriMem: Attribution-Guided Process Feedback for Agent Memory Learning
• Abstract summary: We propose an attribution-guided process-feedback framework for learning memory-construction policies with RL.<n>AttriMem augments the global outcome reward with local rewards derived from token-level contributions to the final answer.
• Score: 13.898789808338634
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Effective memory is crucial for LLM agents, yet constructing it effectively remains challenging. A memory-construction policy decides what information to extract, store, update, compress, or discard as interactions accumulate. Heuristic memory methods rely on subjective, task-specific rules, which can misalign with downstream objectives and limit cross-task adaptability. RL-based methods, by contrast, learn from task feedback but mainly use outcome- or module-level rewards. These coarse signals indicate task success but cannot identify which intermediate memory contents support the final answer, creating a fine-grained credit-assignment bottleneck. However, constructing such process feedback is prohibitively difficult because intermediate memory decisions lack unique ground-truth targets, while the appropriate credit varies with the agent's uncertain reasoning trajectory and therefore cannot be specified in advance. We propose AttriMem, an attribution-guided process-feedback framework for learning memory-construction policies with RL. AttriMem augments the global outcome reward with local rewards derived from token-level contributions to the final answer. Experiments on long-horizon dialogue question answering show that AttriMem outperforms retrieval-based, heuristic, and RL-based baselines, generalizes across benchmarks and answer models, stabilizes RL optimization.

Related papers

• UniMem: Complementary Episodic-to-Parametric Memory for Boundary-Agnostic Task Streams [77.52140119830241]
  We propose UniMem, a self-routing framework for autonomous memory management.<n>UniMem uses learnable routing tokens as memory controllers, enabling adaptive coordination between complementary memory pathways.<n>Experiments on long-horizon streaming task sequences show that UniMem consistently outperforms baselines while maintaining execution fidelity.
  arXiv  Detail & Related papers  (2026-07-28T17:28:21Z)
• MemChain: Learning Interpretable Memory Traces for Memory-Augmented LLM Agents [26.87993979719045]
  Memory-augmented LLM agents typically answer queries by retrieving relevant memories and feeding them directly to an answer model.<n>We propose MemChain, a trainable post-retrieval memory policy that transforms retrieved candidates into answer-facing active memory.<n>We show that MemChain consistently achieves state-of-the-art performance across both closed-source and open-weight frozen answer models.
  arXiv  Detail & Related papers  (2026-07-27T07:37:43Z)
• InfoMem: Training Long-Context Memory Agents with Answer-Conditioned Information Gain [26.098768400948614]
  Long-context tasks require LLMs to identify and preserve answer-relevant information from large contexts.<n>Existing RL-based chunk-wise agents either rely on sparse final-answer rewards or use lexical intermediate rewards for memory and retrieval actions.<n>We propose InfoMem, a reward mechanism for training chunk-wise memory agents that evaluates final-memory utility using answer-conditioned information.
  arXiv  Detail & Related papers  (2026-06-02T08:39:03Z)
• Meta-Cognitive Memory Policy Optimization for Long-Horizon LLM Agents [13.846550145896314]
  Memory-augmented LLM agents tackle complex long-horizon tasks by summarizing interaction trajectories into compact memory.<n>Existing approaches typically train these memory policies using outcome-based reinforcement learning, failing to localize where intermediate memory quality degrades.<n>We argue that memory optimization should focus not merely on trajectory-level success, but on the clarity of the belief induced by intermediate summaries.
  arXiv  Detail & Related papers  (2026-05-28T16:17:19Z)
• Goal-Oriented Reasoning for RAG-based Memory in Conversational Agentic LLM Systems [20.461904943047468]
  Goal-Mem is a goal-oriented reasoning framework for RAG-based agentic memory.<n>We show that Goal-Mem consistently improves performance on tasks requiring multi-hop reasoning and implicit inference.
  arXiv  Detail & Related papers  (2026-05-12T14:51:02Z)
• MemEvoBench: Benchmarking Memory MisEvolution in LLM Agents [78.95081012334116]
  Large Language Models (LLMs) with persistent memory enhances interaction continuity and personalization but introduces new safety risks.<n> contaminated or biased memory accumulation can trigger abnormal agent behaviors.<n>MemeEvoBench is the first benchmark evaluating memory safety in LLM agents against adversarial memory injection, noisy tool outputs, and biased feedback.
  arXiv  Detail & Related papers  (2026-04-17T07:29:52Z)
• Evoking User Memory: Personalizing LLM via Recollection-Familiarity Adaptive Retrieval [59.295767860331004]
  RF-Mem is a familiarity uncertainty-guided dual-path memory retriever.<n>It embeds human-like dual-process recognition into the retriever.<n>It consistently outperforms one-shot retrieval and full-context reasoning under fixed budget and latency constraints.
  arXiv  Detail & Related papers  (2026-03-10T06:31:44Z)
• MemSifter: Offloading LLM Memory Retrieval via Outcome-Driven Proxy Reasoning [78.46301394559903]
  Large Language Models (LLMs) are increasingly used for long-duration tasks.<n>Current methods face a trade-off between cost and accuracy.<n>MemSifter is a novel framework that offloads the memory retrieval process to a small-scale proxy model.
  arXiv  Detail & Related papers  (2026-03-03T02:57:38Z)
• Fine-Mem: Fine-Grained Feedback Alignment for Long-Horizon Memory Management [63.48041801851891]
  Fine-Mem is a unified framework designed for fine-grained feedback alignment.<n> Experiments on Memalpha and MemoryAgentBench demonstrate that Fine-Mem consistently outperforms strong baselines.
  arXiv  Detail & Related papers  (2026-01-13T11:06:17Z)
• MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memory [46.632646462295234]
  We propose MemRL, a framework that enables agents to self-evolve via non-parametric reinforcement learning on episodic memory.<n>MemRL employs a Two-Phase Retrieval mechanism that filters candidates by semantic relevance and then selects them based on learned Q-values.<n>Our analysis experiments confirm that MemRL effectively reconciles the stability-plasticity dilemma, enabling continuous runtime improvement without weight updates.
  arXiv  Detail & Related papers  (2026-01-06T17:14:50Z)
• Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents [57.38404718635204]
  Large language model (LLM) agents face fundamental limitations in long-horizon reasoning due to finite context windows.<n>Existing methods typically handle long-term memory (LTM) and short-term memory (STM) as separate components.<n>We propose Agentic Memory (AgeMem), a unified framework that integrates LTM and STM management directly into the agent's policy.
  arXiv  Detail & Related papers  (2026-01-05T08:24:16Z)
• Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory [89.65731902036669]
  Evo-Memory is a streaming benchmark and framework for evaluating self-evolving memory in large language model (LLM) agents.<n>We evaluate over ten representative memory modules and evaluate them across 10 diverse multi-turn goal-oriented and single-turn reasoning and QA datasets.
  arXiv  Detail & Related papers  (2025-11-25T21:08:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.