When Stored Evidence Stops Being Usable: Scale-Conditioned Evaluation of Agent Memory

• URL: http://arxiv.org/abs/2605.07313v1
• Date: Fri, 08 May 2026 06:22:31 GMT
• Title: When Stored Evidence Stops Being Usable: Scale-Conditioned Evaluation of Agent Memory
• Abstract summary: We present a scale-conditioned evaluation protocol for agent memory under evidence-preserving growth.<n>For each query, task evidence is held fixed while irrelevant sessions are added.<n>The protocol logs agent--memory trajectories and reports four diagnostics.
• Score: 4.425240080125479
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Memory-agent evaluations report fixed-snapshot accuracy or retrieval quality, but these scores do not show whether evidence remains usable as irrelevant sessions (sessions not annotated as task-relevant evidence for the query) accumulate. We present a scale-conditioned evaluation protocol for agent memory under evidence-preserving growth: for each query, task evidence is held fixed while irrelevant sessions are added. The protocol logs agent--memory trajectories and reports four diagnostics: budget-compliant reliability, tail memory-call burden, failure-regime decomposition, and the usable-scale boundary where reliability falls below the target. Applied to LongMemEval and LoCoMo across flat, planar, and hierarchical memory interfaces, the protocol shows reliability loss is not a single phenomenon. On LongMemEval, HippoRAG stays within the two-call budget but loses 16--20 percentage points in budget-compliant reliability as irrelevant sessions are added; LiCoMemory's observed failures depend strongly on the agent, with Qwen3-8B exceeding the budget while Qwen3-32B and Qwen3-235B remain reliable in the tested range. The result supports a framework for making scalable-memory claims conditional on agent, interface, scale range, and interaction budget.

Related papers

• Procedural Memory Under Change: Reuse and Interference in Controlled Web Tasks [0.0]
  Procedural memory lets language agents reuse successful routines, but reuse presumes that a stored routine remains applicable.<n>We study what happens when that presumption is deliberately violated.
  arXiv  Detail & Related papers  (2026-09-09T06:22:42Z)
• What Makes Agent Memory Useful for Reliable Unanswerable Question Handling? [51.58639607746329]
  Memory is widely used in agent systems, but its role in reliable UAQ handling remains unclear.<n>We present a systematic study of agent memory for UAQ handling under a unified agentic RAG framework.<n>We find that memory can improve UAQ performance in some settings, but such gains are selective rather than universal and remain fragile under dataset shift.
  arXiv  Detail & Related papers  (2026-08-28T05:00:28Z)
• D$^2$ACCI: A Dual-Loop Diagnostic Protocol for Evidence-Preserving Agent Memory [10.177699115336425]
  D$2$ACCI protocol promotes, feature-flags, or rejects memory interventions based on paired evidence, protected-slice monitoring, and trace-level localizability.<n>We instantiate the protocol in MemStack and evaluate on three public benchmarks, achieving 93.59% on LoCoMo, 90.93% on LongMemEval, and 57.20% on PersonaMem-V2.
  arXiv  Detail & Related papers  (2026-08-18T13:18:38Z)
• Harness the Memory: A Holistic Evaluation of Memory Substrates in Memory Agents [97.19033877543522]
  We present a controlled harness evaluation of memory substrates for memory-augmented agents.<n>We instrument 26 performance and efficiency metrics under a unified harness.<n>These findings motivate substrate routing as a necessary component of adaptive agent memory systems.
  arXiv  Detail & Related papers  (2026-08-15T03:29:48Z)
• Coupling Planning with Episodic Memory in LLM Agents for Software Issue Resolution [13.491535879221901]
  PMCoder is an issue-resolution agent that couples a hierarchical phase planner with episodic memory.<n>On SWE-bench Verified, PMCoder resolves an average of $25$ more cases than a harness-matched baseline.
  arXiv  Detail & Related papers  (2026-08-07T05:05:05Z)
• MemOps: Benchmarking Lifecycle Memory Operations in Long-Horizon Conversations [50.24315431387575]
  We introduce MemOps, a benchmark that reformulates conversational memory as a sequence of lifecycle operations.<n>MemOps represents each memory event with a structured trace specifying its trigger, target, scope, state transition, and supporting evidence.<n>Across long-context, retrieval-based, parametric and managed-memory systems, MemOps disentangles failure modes that final-answer accuracy alone conceals.
  arXiv  Detail & Related papers  (2026-07-14T15:33:44Z)
• MemRefine: LLM-Guided Compression for Long-Term Agent Memory [67.62350557461352]
  Large language model (LLM) agents are increasingly expected to operate over long-term interactions.<n>As interactions accumulate, the memory store grows without bound and fills with redundant entries that inflate storage cost and degrade retrieval by crowding out the most useful evidence.<n>This is especially limiting on resource-constrained platforms with hard memory budgets.<n>We propose storage-budgeted memory management, the task of keeping an already constructed memory store within a fixed budget while preserving information useful for future interactions.
  arXiv  Detail & Related papers  (2026-06-11T10:46:17Z)
• MemAudit: Post-hoc Auditing of Poisoned Agent Memory via Causal Attribution and Structural Anomaly Detection [15.403262505444454]
  We propose a post-hoc causal memory auditing framework for large language model agents.<n>MemAudit substantially reduces attack success rates under realistic post-hoc auditing scenarios.
  arXiv  Detail & Related papers  (2026-05-22T15:03:13Z)
• Belief Memory: Agent Memory Under Partial Observability [56.41506249481312]
  We propose BeliefMem, which shifts the memory paradigm from committing to a single conclusion per observation to retaining multiple candidate conclusions with their probabilities.<n> BeliefMem preserves the uncertainty that the deterministic paradigm discards, enabling the agent to act with high confidence.<n> Empirical evaluations on LoCoMo and ALFWorld benchmarks show that, even with limited data, BeliefMem achieves the best average performance.
  arXiv  Detail & Related papers  (2026-05-07T02:03:13Z)
• MemEvoBench: Benchmarking Memory MisEvolution in LLM Agents [78.95081012334116]
  Large Language Models (LLMs) with persistent memory enhances interaction continuity and personalization but introduces new safety risks.<n> contaminated or biased memory accumulation can trigger abnormal agent behaviors.<n>MemeEvoBench is the first benchmark evaluating memory safety in LLM agents against adversarial memory injection, noisy tool outputs, and biased feedback.
  arXiv  Detail & Related papers  (2026-04-17T07:29:52Z)
• All-Mem: Agentic Lifelong Memory via Dynamic Topology Evolution [20.68235182577703]
  All-Mem maintains a structured memory bank via explicit, non destructive consolidation.<n>Periodically offline, an LLM diagnoser proposes confidence scored topology edits executed with three operators: SPLIT, MERGE, and UPDATE.<n>At query time, typed links enable hop bounded, budgeted expansion from active anchors to archived evidence when needed.
  arXiv  Detail & Related papers  (2026-03-20T03:14:40Z)
• MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks [55.145729491377374]
  Existing evaluations of agents with memory typically assess memorization and action in isolation.<n>We introduce MemoryArena, a unified evaluation gym for benchmarking agent memory in multi-session Memory-Agent-Environment loops.<n> MemoryArena supports evaluation across web navigation, preference-constrained planning, progressive information search, and sequential formal reasoning.
  arXiv  Detail & Related papers  (2026-02-18T09:49:14Z)
• Decomposed Prompting Does Not Fix Knowledge Gaps, But Helps Models Say "I Don't Know" [47.930782177987446]
  Large language models often struggle to recognize their knowledge limits in closed-book question answering, leading to confident hallucinations.<n>We evaluate three task-equivalent prompting regimes: Direct, Assistive, and Incremental, across different model scales and multi-hop QA benchmarks.<n>Because factual knowledge is stable while hallucinations are agreement, cross-regime provides a precise signal of internal uncertainty.
  arXiv  Detail & Related papers  (2026-02-04T18:39:58Z)
• Fine-Mem: Fine-Grained Feedback Alignment for Long-Horizon Memory Management [63.48041801851891]
  Fine-Mem is a unified framework designed for fine-grained feedback alignment.<n> Experiments on Memalpha and MemoryAgentBench demonstrate that Fine-Mem consistently outperforms strong baselines.
  arXiv  Detail & Related papers  (2026-01-13T11:06:17Z)
• PBFT-Backed Semantic Voting for Multi-Agent Memory Pruning [0.0]
  This paper introduces the Co-Forgetting Protocol to enable synchronized memory pruning in multi-agent systems (MAS)<n>The protocol integrates three key components: (1) context-aware voting, where agents utilize a lightweight DistilBERT model to assess the relevance of memory items based on their content and the current operational context; (2) multi-scale temporal decay functions, which assign diminishing importance to memories based on their age and access frequency across different time horizons; and (3) a Practical Byzantine Fault Tolerance (PBFT)-based consensus mechanism, ensuring that decisions to retain or discard memory items are agreed upon by a qualified and fault-tolerant majority of agents
  arXiv  Detail & Related papers  (2025-06-19T08:28:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.