Reclaim Evaluation: A Lossy Memory Is Worse Than an Empty One

• URL: http://arxiv.org/abs/2606.25449v1
• Date: Wed, 24 Jun 2026 06:24:53 GMT
• Title: Reclaim Evaluation: A Lossy Memory Is Worse Than an Empty One
• Abstract summary: A language model's memory can be worse than having no memory at all.<n>A single dropped-source error corrupts a growing span of downstream steps and stays uncorrectable.<n>This is a controlled study of a mechanism, not a benchmark.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A language model's memory can be worse than having no memory at all. Give a model a memory that kept a wrong conclusion but dropped the work behind it, and it emits that stale value as a confident answer; give the same model an empty memory and it abstains. Across seven models this direction never reverses, a clean kill condition that none breaks. We call this brittle memory: behavioral, not the near-immediate information bound beneath it; only its magnitude is disposition- and task-dependent, not its direction. We measure it with reclaim evaluation: compress a drifted interaction at a fixed budget, then test whether a correction recovers the known answer, scored against ground truth with no judge. Correctability is bottlenecked by whether the answer-determining source survives, not by capability. A one-line source-first policy (keep the recomputable source, drop the re-derivable conclusion) restores correctability at equal budget where that source is compact and identifiable; a length-matched control rules out added text as the cause. The hand-built oracle reaches 1.00; a one-prompt deployable version reclaims 0.49-0.88. The stake compounds: chained through a memory loop, a single dropped-source error corrupts a growing span of downstream steps and stays uncorrectable, while source-first holds to a bounded budget horizon. The wall and fix replicate across three deployed memory systems and on real dialogue (MultiWOZ), and past the budget where the source no longer fits, the fix fails silently unless the note records completeness. This is a controlled study of a mechanism, not a benchmark: judge-free exact scoring, matched-budget controls, and validators built to come out false. We release the harness, conditions, and validators.

Related papers

• The Cost of Compression: A Rate-Distortion Limit on Factual Hallucination [5.158280959759542]
  Factual hallucination in closed-book question answering is often treated as a coverage problem.<n>We study this effect through a simple coverage--compression model of factual recall.<n>We study the predicted signatures with theory-implied simulations and controlled fact-injection probes in modern language models.
  arXiv  Detail & Related papers  (2026-09-10T18:37:44Z)
• Towards a Formal Definition of Agent Memory: Basis, Span, Optimality, and the Sequential Memory Problem [12.078734827417685]
  There is no unified formal account of what a memory is or when it is optimal.<n>The central idea is that memory is a basis, knowledge is its span, and answerability is a coverage problem.<n>The optimal memory is then the capacity-constrained maximizer of expected coverage.
  arXiv  Detail & Related papers  (2026-08-12T04:54:26Z)
• From Faulty Memories to Corrected Actions: Dependency-Guided Rollback Repair for Memory-Augmented Agents [19.125716338799545]
  textbfdependency-guided rollback repair builds a typed memory-to-action graph from provenance, traces explicit downstream dependencies, and selectively replays only answer-relevant affected computation.<n>We evaluate this approach on a 150-case controlled benchmark spanning three tool-use domains and four memory failure types.
  arXiv  Detail & Related papers  (2026-08-11T05:19:55Z)
• Voice Memory for Agentic Speech Recognition [66.69623133635868]
  We present Voice Memory, a inference-only scheme for agentic speech recognition.<n>Asynchronously, a score-gated example revises that file through bounded edits.<n>Across ten HyPoradise domains with an open corrector, Voice Memory, weighted word error rate from 8.36% to 7.52%.
  arXiv  Detail & Related papers  (2026-07-29T02:42:56Z)
• Keep It InMind: Benchmarking the Implicit-Association Blind Spot in Agent Memory [32.094723684433895]
  Long-term memory systems store what a user says in an external store and retrieve it when a related query arrives.<n>This interface rests on an assumption so natural that it is rarely stated: a memory that is needed will resemble the query that needs it.<n>We call this failure mode the implicit-association blind spot and introduce InMind, a benchmark spanning ten life domains.
  arXiv  Detail & Related papers  (2026-07-27T12:42:12Z)
• Forget to Improve: On-Device LLM-Agent Continual Learning via Budget-Curated Memory [7.254575731198877]
  On-device language-model agents improve by accumulating experience in retrieved memory rather than by updating weights.<n>We propose sys, a single net-value-per-byte score that governs an agent's experience-memory lifecycle.
  arXiv  Detail & Related papers  (2026-06-23T19:42:07Z)
• Learning What to Remember: A Cognitively Grounded Multi-Factor Value Model for Agentic Memory [1.2127875744950842]
  Long-running LLM agents accumulate interaction histories far larger than any context window.<n>Production systems answer with semantic similarity or recency.<n>We propose a multi-factor memory function V(m)=sum_i w_i f_i(m) over seven interpretable factors.
  arXiv  Detail & Related papers  (2026-06-11T06:17:31Z)
• Echo-Memory: A Controlled Study of Memory in Action World Models [58.77172260403133]
  We present a controlled study of memory mechanisms in action-conditioned world models.<n>Existing memory designs are hard to compare because gains are entangled with backbone, training, retrieval, and evaluation differences.<n>We use Echo-Memory to fix the action-to-video interface and varies only how history is stored and read by the generator.
  arXiv  Detail & Related papers  (2026-06-08T17:54:10Z)
• Precision Is Not Faithfulness: Coverage-Aware Evaluation of Grounded Generation with a Complete Oracle [0.0]
  Reference-free faithfulness metrics verify each atomic claim a model makes against ground truth.<n>We show they share a blind spot: they measure only precision, are the stated claims supported?<n>We make this measurable using Formula 1 telemetry, a domain where strategic ground truth is derived deterministically.
  arXiv  Detail & Related papers  (2026-06-08T11:56:25Z)
• TOKI: A Bitemporal Operator Algebra for Contradiction Resolution in LLM-Agent Persistent Memory [22.85129722207174]
  We show that contradiction resolution is write-time control and make the missing contract explicit.<n> TOKI types the fourjudicats as one family of bitemporal operators over a dual-row schema.<n>A verdict matrix over eight systems localizes the gap: every baseline that keeps a language-model judge on the write path admits at least one of three write-time anomalies.
  arXiv  Detail & Related papers  (2026-06-04T14:46:52Z)
• WorldMemArena: Evaluating Multimodal Agent Memory Through Action-World Interaction [72.1620416874118]
  Multimodal large language models are increasingly deployed as long-horizon agents.<n>Existing benchmarks measure recall over static dialogue, collapse memory into a single end-of-task accuracy, and reduce visual observations to captions.<n>We formulate multimodal agent memory as an Action-World Interaction Loop with an observable four-stage lifecycle.
  arXiv  Detail & Related papers  (2026-05-28T04:27:20Z)
• MemGuard: Preventing Memory Contamination in Long-Term Memory-Augmented Large Language Models [56.31411457917676]
  We introduce MemGuard, a type-aware memory framework that preserves functional memory boundaries during memory construction and retrieval.<n>Across hallucination and long-horizon conversation benchmarks, MemGuard improves memory reliability by up to 28.27% while retrieving up to 5.8x fewer memory tokens than prior methods.
  arXiv  Detail & Related papers  (2026-05-27T06:04:19Z)
• Remember the Decision, Not the Description: A Rate-Distortion Framework for Agent Memory [120.68750043918249]
  We show that memory should preserve the distinctions that matter for decisions, not descriptions.<n>We propose DeMem, an online memory learner that refines its partition only when data certify that a shared state would induce decision conflict.
  arXiv  Detail & Related papers  (2026-05-11T17:20:58Z)
• When Stored Evidence Stops Being Usable: Scale-Conditioned Evaluation of Agent Memory [4.425240080125479]
  We present a scale-conditioned evaluation protocol for agent memory under evidence-preserving growth.<n>For each query, task evidence is held fixed while irrelevant sessions are added.<n>The protocol logs agent--memory trajectories and reports four diagnostics.
  arXiv  Detail & Related papers  (2026-05-08T06:22:31Z)
• Belief Memory: Agent Memory Under Partial Observability [56.41506249481312]
  We propose BeliefMem, which shifts the memory paradigm from committing to a single conclusion per observation to retaining multiple candidate conclusions with their probabilities.<n> BeliefMem preserves the uncertainty that the deterministic paradigm discards, enabling the agent to act with high confidence.<n> Empirical evaluations on LoCoMo and ALFWorld benchmarks show that, even with limited data, BeliefMem achieves the best average performance.
  arXiv  Detail & Related papers  (2026-05-07T02:03:13Z)
• Evoking User Memory: Personalizing LLM via Recollection-Familiarity Adaptive Retrieval [59.295767860331004]
  RF-Mem is a familiarity uncertainty-guided dual-path memory retriever.<n>It embeds human-like dual-process recognition into the retriever.<n>It consistently outperforms one-shot retrieval and full-context reasoning under fixed budget and latency constraints.
  arXiv  Detail & Related papers  (2026-03-10T06:31:44Z)
• Remember Me, Refine Me: A Dynamic Procedural Memory Framework for Experience-Driven Agent Evolution [52.76038908826961]
  We propose $textbfReMe$ ($textitRemember Me, Refine Me$) to bridge the gap between static storage and dynamic reasoning.<n>ReMe innovates across the memory lifecycle via three mechanisms: $textitmulti-faceted distillation$, which extracts fine-grained experiences.<n>Experiments on BFCL-V3 and AppWorld demonstrate that ReMe establishes a new state-of-the-art in agent memory system.
  arXiv  Detail & Related papers  (2025-12-11T14:40:01Z)
• Agentic Learner with Grow-and-Refine Multimodal Semantic Memory [50.81667005063605]
  ViLoMem is a dual-stream memory framework that constructs compact, schema-based memory.<n>It encodes visual distraction patterns and logical reasoning errors, enabling MLLMs to learn from their successful and failed experiences.
  arXiv  Detail & Related papers  (2025-11-26T18:55:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.