Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents

• URL: http://arxiv.org/abs/2606.27472v1
• Date: Thu, 25 Jun 2026 18:50:32 GMT
• Title: Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents
• Abstract summary: Large language model (LLM) agents operate over long, multi-session interactions in which facts change.<n>We show that acting correctly requires using the current value of a fact and discarding values that have been superseded.<n>We release Supersede, an open reinforcement-learning environment that turns this measurement into a training signal.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language model (LLM) agents operate over long, multi-session interactions in which facts change: a user moves, a price updates, a plan is revised. Acting correctly requires using the current value of a fact and discarding values that have been superseded. We isolate this ability on real conversational data and show that it is a distinct, unsolved failure. On the knowledge-update subset of LongMemEval, replacing an agent's full context with a bounded, self-maintained memory drops accuracy from 92% to 77% even on a frontier model (gpt-5.4), a gap that is statistically significant (paired McNemar p<0.005) and persists across model scale while full-context accuracy saturates near 92%. The bottleneck is therefore memory maintenance, not comprehension, and is not closed by a stronger model. We then ask whether this is merely an undersized memory, and find it is not: as the conversation grows 24x, accuracy falls further (from 68% to 28%), and granting the agent proportionally more memory yields no detectable recovery (28% to 28%, n=25). The failure scales with the length of the conversation, not the compression ratio. We release Supersede, an open reinforcement-learning environment (on the verifiers / prime-rl stack) that turns this measurement into a training signal: agents are rewarded for answering from the current value and penalized for stale ones. Finally, we close the loop and show the gap is trainable: GRPO fine-tuning a small open model (Qwen2.5-3B) on this environment nearly doubles its held-out supersession accuracy on real, unseen conversations (9.0% to 16.7%, a single run), along a monotonic checkpoint curve indicating the learned policy, not the harness, carries the gain. To our knowledge this is the first trainable environment whose reward targets temporal fact-currency, and the first evidence the supersession gap can be trained down, not only measured.

Related papers

• ReDraft, Don't Just Distill: Reference-Driven Revision for Continual VLLM Post-Training [66.72708893922605]
  Post-training of large multimodal models should add new capabilities while preserving those from pre-training.<n>SFT gives explicit target supervision that learns a task from near-zero accuracy, but its off-policy targets move the model far enough to cause forgetting.<n>We introduce ReDraft, which obtains both from the model's own failures.
  arXiv  Detail & Related papers  (2026-09-15T04:59:03Z)
• The Memory Trust Gap: Capability-Dependent Failures in Persistent-Memory Agents [0.0]
  Persistent memory supports personalized agents, but a stale stored fact can override current authoritative evidence without warning.<n>We study when this harm begins as model capability changes.
  arXiv  Detail & Related papers  (2026-09-01T20:35:00Z)
• Temporal Validity in Retrieval Memory: Eliminating Stale-Fact Errors for AI Agents over Evolving Knowledge [0.0]
  Retrieval-augmented generation gives agents access to accumulated knowledge, but has no model of time.<n>We present MemStrata, a retrieval memory maintaining temporal validity.<n>Across six benchmarks run locally with a 7B model, MemStrata ties RAG on static knowledge and reaches 0.95-1.00 accuracy on evolving knowledge.
  arXiv  Detail & Related papers  (2026-06-25T01:31:53Z)
• Learning What to Remember: A Cognitively Grounded Multi-Factor Value Model for Agentic Memory [1.2127875744950842]
  Long-running LLM agents accumulate interaction histories far larger than any context window.<n>Production systems answer with semantic similarity or recency.<n>We propose a multi-factor memory function V(m)=sum_i w_i f_i(m) over seven interpretable factors.
  arXiv  Detail & Related papers  (2026-06-11T06:17:31Z)
• Fast Unlearning at Scale via Margin Self-Correction [52.46927918952516]
  Language-model unlearning updates a trained model to behave as if it had not seen selected training examples.<n>MASC achieves a competitive forget-retain trade-off at a fraction of the computational cost of existing baselines.
  arXiv  Detail & Related papers  (2026-06-01T21:49:54Z)
• Understanding and Mitigating Premature Confidence for Better LLM Reasoning [76.16007941549857]
  Long chains of thought (CoT) from current language models frequently contain logical gaps and unjustified leaps.<n>We find such a signal in how the model's confidence evolves during reasoning.<n>We exploit this in progressive confidence shaping, a reinforcement learning objective that trains models to update their confidence as they reason rather than commit early.
  arXiv  Detail & Related papers  (2026-05-23T04:42:45Z)
• Learn by Surprise, Commit by Proof [0.0]
  We propose a self-gated post-training framework for autonomous knowledge acquisition.<n>When a passage produces anomalously high pertoken loss, L SCP flags it, generates a Q&A chain that forces the model to articulate its own knowledge.<n>The entire learning intensity is governed by a single parameter $r$.
  arXiv  Detail & Related papers  (2026-04-02T12:17:10Z)
• Knowledge Access Beats Model Size: Memory Augmented Routing for Persistent AI Agents [9.457255218406333]
  Production AI agents frequently receive user-specific queries that are highly repetitive, with up to 47% semantically similar to prior interactions.<n>We argue that this redundancy can be exploited through conversational memory, transforming repetition from a cost burden into an efficiency advantage.
  arXiv  Detail & Related papers  (2026-03-24T09:55:11Z)
• When to Memorize and When to Stop: Gated Recurrent Memory for Long-Context Reasoning [50.486479460454866]
  We propose GRU-Mem, which incorporates two text-controlled gates for more stable and efficient long-context reasoning.<n>GRU-Mem generally outperforms the vanilla MemAgent with up to 400% times inference speed acceleration.
  arXiv  Detail & Related papers  (2026-02-11T06:14:53Z)
• Compact Memory for Continual Logistic Regression [45.6774409797545]
  We present a new method to build compact memory for logistic regression.<n>Our approach can drastically improve accuracy compared to Experience Replay.<n>Our work opens a new direction for building compact memory that can also be useful in the future for continual deep learning.
  arXiv  Detail & Related papers  (2025-11-12T10:05:06Z)
• Train for Truth, Keep the Skills: Binary Retrieval-Augmented Reward Mitigates Hallucinations [103.16279860448874]
  We propose an online reinforcement learning method using a novel binary retrieval-augmented reward (RAR)<n>For open-ended generation, binary RAR achieves a 39.3% reduction in hallucination rates.<n>In short-form question answering, the model learns abstention, strategically outputting "I don't know" when faced with insufficient parametric knowledge.
  arXiv  Detail & Related papers  (2025-10-20T16:45:43Z)
• SimpleToM: Exposing the Gap between Explicit ToM Inference and Implicit ToM Application in LLMs [72.06808538971487]
  We test whether large language models (LLMs) can implicitly apply a "theory of mind" (ToM) to predict behavior. We create a new dataset, SimpleTom, containing stories with three questions that test different degrees of ToM reasoning. To our knowledge, SimpleToM is the first dataset to explore downstream reasoning requiring knowledge of mental states in realistic scenarios.
  arXiv  Detail & Related papers  (2024-10-17T15:15:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.