MemTrace: Tracing and Attributing Errors in Large Language Model Memory Systems

• URL: http://arxiv.org/abs/2605.28732v1
• Date: Wed, 27 May 2026 16:53:53 GMT
• Title: MemTrace: Tracing and Attributing Errors in Large Language Model Memory Systems
• Abstract summary: We study the new problem of error tracing and attribution in LLM memory systems.<n>We propose a novel framework that transforms memory pipelines into executable memory evolution graphs.<n>We then construct MemTraceBench, a benchmark collected from memory systems such as Long-Context, RAG, Mem0, and EverMemOS.
• Score: 24.22940060094778
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Memory is essential for enabling large language models to support long-horizon reasoning, yet existing memory systems remain unreliable and difficult to debug. Tracing memory's dynamic evolution is crucial to understand how information is synthesized, propagated, or corrupted over time. In this work, we study the new problem of error tracing and attribution in LLM memory systems. We propose a novel framework that transforms memory pipelines into executable memory evolution graphs, enabling fine-grained tracing of operational information flow. We then construct MemTraceBench, a benchmark collected from representative memory systems such as Long-Context, RAG, Mem0, and EverMemOS, to systematically study memory failure modes. We further introduce an automatic attribution method that iteratively traces operation subgraphs to pinpoint the root cause of any failed case. Our analysis reveals that memory failures are systematic, stemming from operation-level issues like information loss and retrieval misalignment. Crucially, we leverage these fine-grained attribution signals to guide downstream prompt optimization, establishing a closed-loop system that automatically corrects faults and boosts end-task performance by up to 7.62%. Code will be released at https://github.com/zjunlp/MemTrace.

Related papers

• InjecMEM: Memory Injection Attack on LLM Agent Memory Systems [31.104106193922956]
  InjecMEM is a novel memory injection attack paradigm.<n>We craft the injection with a retriever-agnostic anchor and an adversarial command.<n>We evaluate it across multiple memory systems and backbone models.
  arXiv  Detail & Related papers  (2026-08-24T16:37:50Z)
• Mechanistic Attention Guidance for Agent Memory Refinement [32.57216046158823]
  We show that retrieval-head attention provides a mechanistic signal for revealing segment-level memory utilization.<n>We propose Attention-Guided Memory Refinement (AGMR), a framework that uses utilization patterns revealed by attention to guide targeted segment-level memory updates.
  arXiv  Detail & Related papers  (2026-07-20T07:17:50Z)
• Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents [72.3752981234916]
  Large Language Model (LLM) agents increasingly rely on external memory systems to accumulate experience across tasks.<n>We present Memory as a Controlled Process (MemCon), a framework that models memory operations as a Markov Decision Process.<n>MemCon consistently outperforms multiple memory baselines by up to 15.2 points in task success while reducing token consumption by 5-20%.
  arXiv  Detail & Related papers  (2026-07-15T08:32:40Z)
• MemPro: Agentic Memory Systems as Evolvable Programs [26.76706603895889]
  Long-horizon autonomous agents require memory systems to retain historical information, track evolving states, and reuse relevant knowledge beyond finite context windows.<n>Existing agentic memory systems typically follow a memory construction-retrieval (MCR) pipeline, but often adapt mainly the memory bank while keeping the surrounding pipeline fixed after deployment.<n>We propose MemPro, a system-level evolution framework that treats the entire MCR pipeline as an evolvable program rather than adapting only the memory bank or prompt text.
  arXiv  Detail & Related papers  (2026-05-30T08:47:33Z)
• MemFail: Stress-Testing Failure Modes of LLM Memory Systems [69.80981631587501]
  Large language model (LLM) agents increasingly rely on external memory systems to remain consistent across long-horizon interactions.<n>Existing benchmarks report aggregate question-answering accuracy and treat memory systems as black boxes.<n>We introduce MemFail, a diagnostic benchmark that isolates the failure modes of modern LLM memory systems.
  arXiv  Detail & Related papers  (2026-05-26T08:03:55Z)
• Causal Intervention-Based Memory Selection for Long-Horizon LLM Agents [0.33842793760651557]
  We propose a causal memory-selection technique that estimates how candidate memories affect the model's answer under controlled interventions.<n>Results show that CMI achieves a stronger balance between answer quality and robustness to misleading memory, suggesting that reliable long-term memory requires selecting context based on causal usefulness rather than relevance alone.
  arXiv  Detail & Related papers  (2026-05-17T20:21:55Z)
• MemCompiler: Compile, Don't Inject -- State-Conditioned Memory for Embodied Agents [29.872311031034368]
  We propose MemCompiler, which reframes memory utilization as State-Conditioned Memory Compilation.<n>Across Alf World, EmbodiedBench, and ScienceWorld, MemCompiler consistently improves over no-memory.
  arXiv  Detail & Related papers  (2026-05-08T11:07:04Z)
• MemEvoBench: Benchmarking Memory MisEvolution in LLM Agents [78.95081012334116]
  Large Language Models (LLMs) with persistent memory enhances interaction continuity and personalization but introduces new safety risks.<n> contaminated or biased memory accumulation can trigger abnormal agent behaviors.<n>MemeEvoBench is the first benchmark evaluating memory safety in LLM agents against adversarial memory injection, noisy tool outputs, and biased feedback.
  arXiv  Detail & Related papers  (2026-04-17T07:29:52Z)
• MemMA: Coordinating the Memory Cycle through Multi-Agent Reasoning and In-Situ Self-Evolution [52.29303869050117]
  Memory-augmented LLM agents maintain external memory banks to support long-horizon interaction.<n>MemMA is a plug-and-play multi-agent framework that coordinates the memory cycle along both the forward and backward paths.
  arXiv  Detail & Related papers  (2026-03-19T10:15:59Z)
• NextMem: Towards Latent Factual Memory for LLM-based Agents [58.35585202907478]
  NextMem is a latent factual memory framework that utilizes an autoregressive autoencoder to efficiently construct latent memory.<n>Extensive experiments demonstrate that NextMem achieves superior performance.
  arXiv  Detail & Related papers  (2026-02-26T14:35:27Z)
• MemOS: A Memory OS for AI System [116.87568350346537]
  Large Language Models (LLMs) have become an essential infrastructure for Artificial General Intelligence (AGI)<n>Existing models mainly rely on static parameters and short-lived contextual states, limiting their ability to track user preferences or update knowledge over extended periods.<n>MemOS is a memory operating system that treats memory as a manageable system resource.
  arXiv  Detail & Related papers  (2025-07-04T17:21:46Z)
• MemOS: An Operating System for Memory-Augmented Generation (MAG) in Large Language Models [31.944531660401722]
  We introduce MemOS, a memory operating system designed for Large Language Models (LLMs)<n>At its core is the MemCube, a standardized memory abstraction that enables tracking, fusion, and migration of heterogeneous memory.<n>MemOS establishes a memory-centric execution framework with strong controllability, adaptability, and evolvability.
  arXiv  Detail & Related papers  (2025-05-28T08:27:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.