Does Video Memory Use What It Retrieves? A Causal Audit of Memory Specificity

• URL: http://arxiv.org/abs/2609.12090v1
• Date: Thu, 10 Sep 2026 18:16:32 GMT
• Title: Does Video Memory Use What It Retrieves? A Causal Audit of Memory Specificity
• Abstract summary: Video models increasingly use memory to preserve information over long sequences.<n>Standard memory ablations test whether memory helps, but not whether the retrieved content is responsible.<n>We test this directly with read-time memory substitution.
• Score: 38.098764596232996
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video models increasingly use memory to preserve information over long sequences, with the assumption that gains come from retrieving and using the correct past content. Standard memory ablations test whether memory helps, but not whether the retrieved content is responsible. We test this directly with read-time memory substitution, which replaces the consumed memory value while leaving the rest of the computation unchanged. This separates memory benefit from memory specificity, the extent to which the gain depends on retrieved content. Across frozen video world models, identity-free controls containing no evaluation-specific content recover essentially the full benefit on Ego-Exo4D and 7-Scenes and about 70% on TUM. In the Ego-Exo4D dose response, recovery falls from 102% to 1% as these values move away from observed training-memory representations, supporting representation repair as the best-supported explanation in this setting. WorldMem shows graded dependence. A wrong memory from the same trajectory recovers 94.1% of the PSNR benefit relative to zero content, while a donor from a disjoint trajectory and biome recovers 43.7%. SAM 2 shows strong content dependence. On DAVIS, replacing the correct spatial memory with a valid wrong memory reduces mean region and boundary score from 0.926 to 0.182. At MOSEv2 reappearance, it falls from 0.459 to 0.000. These results show that memory gains can depend on generic representation support, broader context, or exact episodic content. Read-time substitution provides a direct way to distinguish them.

Related papers

• Where Should Experience Live? Hierarchical Hebbian Memory for Continual Vision Transformers [0.0]
  Vision Transformers provide strong visual representations but typically rely on slowly updated parameters.<n>This work proposes textitHierarchical Hebbian Memory, a three-level memory architecture composed of rapid Working Memory, persistent Routed Episodic Memory, and slower Semantic Memory.
  arXiv  Detail & Related papers  (2026-08-31T20:51:51Z)
• From Faulty Memories to Corrected Actions: Dependency-Guided Rollback Repair for Memory-Augmented Agents [19.125716338799545]
  textbfdependency-guided rollback repair builds a typed memory-to-action graph from provenance, traces explicit downstream dependencies, and selectively replays only answer-relevant affected computation.<n>We evaluate this approach on a 150-case controlled benchmark spanning three tool-use domains and four memory failure types.
  arXiv  Detail & Related papers  (2026-08-11T05:19:55Z)
• MemSyco-Bench: Benchmarking Sycophancy in Agent Memory [32.49512092540906]
  MemSyco-Bench is a benchmark for evaluating memory-induced sycophancy in agent systems.<n>MemSyco-Bench measures when memory should influence a decision and how valid memory should be used.
  arXiv  Detail & Related papers  (2026-07-01T15:30:33Z)
• MEMPROBE: Probing Long-Term Agent Memory via Hidden User-State Recovery [58.64329475683699]
  We argue that long-term memory should be evaluated as an auditable post-interaction artifact.<n>We instantiate this view in MEMPROBE, a benchmark in which a memory-equipped agent assists simulated users.<n>We find that successful assistance and recoverable memory behave as distinct capabilities.
  arXiv  Detail & Related papers  (2026-06-23T13:52:46Z)
• Deployment-Time Memorization in Foundation-Model Agents [18.40535884093806]
  We study agent memory as a privacy-utility frontier measured by Personalization Recall (PR) and Adrial Extraction Rate (AER)<n>Key-fact summarization reduces canary extraction by 76% on Gemma 3 12B and 64% on GPT-4o-mini while preserving nearly all personalization recall.<n>On LongMemEval, key-fact summarization reduces canary extraction by 76% on Gemma 3 12B and 64% on GPT-4o-mini while preserving nearly all personalization recall.
  arXiv  Detail & Related papers  (2026-06-08T18:38:41Z)
• Echo-Memory: A Controlled Study of Memory in Action World Models [58.77172260403133]
  We present a controlled study of memory mechanisms in action-conditioned world models.<n>Existing memory designs are hard to compare because gains are entangled with backbone, training, retrieval, and evaluation differences.<n>We use Echo-Memory to fix the action-to-video interface and varies only how history is stored and read by the generator.
  arXiv  Detail & Related papers  (2026-06-08T17:54:10Z)
• When Should Memory Stay Silent: Measuring Memory-Use Boundaries in Memory-Augmented Conversational Agents [13.354128891280316]
  Long-term memory enables language model agents to support personalized interactions.<n>It remains unclear when available memories warrant integration into responses.<n>Retrieval systems reduce exposure but do not eliminate integration once sensitive memory reaches the generator.
  arXiv  Detail & Related papers  (2026-06-04T11:54:51Z)
• MemGuard: Preventing Memory Contamination in Long-Term Memory-Augmented Large Language Models [56.31411457917676]
  We introduce MemGuard, a type-aware memory framework that preserves functional memory boundaries during memory construction and retrieval.<n>Across hallucination and long-horizon conversation benchmarks, MemGuard improves memory reliability by up to 28.27% while retrieving up to 5.8x fewer memory tokens than prior methods.
  arXiv  Detail & Related papers  (2026-05-27T06:04:19Z)
• Is One Score Enough? Rethinking the Evaluation of Sequentially Evolving LLM Memory [50.857546269660276]
  We introduce SeqMem-Eval, a diagnostic evaluation framework for sequentially evolving large language models (LLMs) memory.<n>Rather than focusing only on final performance, SeqMem-Eval evaluates how memory states evolve, generalize, consolidate experience, and retain useful information during sequential inference.
  arXiv  Detail & Related papers  (2026-05-14T20:15:22Z)
• Recurrent Dynamic Embedding for Video Object Segmentation [54.52527157232795]
  We propose a Recurrent Dynamic Embedding (RDE) to build a memory bank of constant size. We propose an unbiased guidance loss during the training stage, which makes SAM more robust in long videos. We also design a novel self-correction strategy so that the network can repair the embeddings of masks with different qualities in the memory bank.
  arXiv  Detail & Related papers  (2022-05-08T02:24:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.