Eviction as Estimation: A Fixed-Lag Smoothing View of Test-Time Memory, and When Measuring Beats Accumulating

• URL: http://arxiv.org/abs/2607.24667v1
• Date: Mon, 27 Jul 2026 17:08:27 GMT
• Title: Eviction as Estimation: A Fixed-Lag Smoothing View of Test-Time Memory, and When Measuring Beats Accumulating
• Abstract summary: A language model with a bounded working memory must repeatedly decide which stored items to keep.<n>We instantiate it as a training-free policy, RMM, a strict generalization of H2O.<n>But on independent benchmarks, run inside NVIDIA's KVPress harness against its own SnapKV, H2O, and StreamingLLM implementations, the advantage mostly disappears.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A language model with a bounded working memory must repeatedly decide which stored items to keep. Every deployed method decides the moment an item arrives, from the past (StreamingLLM, H2O) or from a guess about the future (SnapKV). We recast the choice as an estimation problem on a hidden signal, whether an item will be reused, placing existing methods on one axis, the commit lag $H$: online filters and learned predictors commit at $H=0$, while Belady's offline optimum sits where the whole future is known. The missing regime in between, fixed-lag smoothing, waits a bounded number of steps, observes which items a correct near-future prediction attended to, and only then commits. This measurement, demonstrated utility, turns Belady's unobservable future request into something we read off the model itself. We instantiate it as a training-free policy, RMM, a strict generalization of H2O that reduces to it exactly when the measurement is uniform. In controlled settings where reuse is endogenous and separated in time, demonstrated utility identifies used memory far better than accumulated attention, and a small bounded memory behaves like a much larger one. But on independent third-party benchmarks, run inside NVIDIA's KVPress harness against its own SnapKV, H2O, and StreamingLLM implementations, the advantage mostly disappears: RMM is on par with H2O for single-turn question answering and loses to both H2O and SnapKV in a streaming multi-turn setting. The cause is simple: on natural text the model is correct about most tokens, so weighting attention by correctness barely changes it, and demonstrated utility collapses onto accumulated attention unless reuse is sharp and endogenous, which standard benchmarks do not exercise. Our contribution is the framework and an honest map of when measuring beats accumulating, not a new state of the art.

Related papers

• The Memory Trust Gap: Capability-Dependent Failures in Persistent-Memory Agents [0.0]
  Persistent memory supports personalized agents, but a stale stored fact can override current authoritative evidence without warning.<n>We study when this harm begins as model capability changes.
  arXiv  Detail & Related papers  (2026-09-01T20:35:00Z)
• When the Martingale Never Stops Firing: Anytime-Valid Gating on Real Forecast Streams [0.614481021961242]
  Anytime-valid inference promises evidence that can be acted on at any moment.<n>A deployment inherits it only if the stream it monitors behaves exchangeably.
  arXiv  Detail & Related papers  (2026-08-31T09:32:11Z)
• Interaction Scaling: Grounding the Third Axis of Test-Time Compute [0.33451037881913753]
  We argue that a single variable governs this third axis, grounding, and that it must hold on both sides of the loop.<n>On hard coding tasks at a fixed token budget, reasoning-only and best-of-N sampling both plateau.<n>Our proposer-reviewer harness reaches a perfect 100% pass rate with no run-to-run variance.
  arXiv  Detail & Related papers  (2026-07-13T14:22:06Z)
• Remembering Distinct Items, Not Tokens: A Learnable Dirichlet-Process Cache Between State-Space Models and Attention [0.48342038441006796]
  We study a sparse cache that allocates a slot only when an input is novel, so its size tracks the number of distinct items rather than the number of tokens.<n>We develop it in two forms, a static cache with a fixed concentration and a surprise-adaptive variant whose concentration follows the recent novelty rate.
  arXiv  Detail & Related papers  (2026-07-10T18:28:21Z)
• What to Keep, What to Forget: A Rate--Distortion View of Memory Compaction in LLMs and Agents [1.0742675209112622]
  We show how four separate research communities have each learned to compact large language memory.<n>We argue that these are instances of one problem: a rate--distortion decision about what context-derived information to retain versus discard.<n>We make this lens precise with a single compaction objective and a layer-agnostic lower bound, use it to build a seven-axis taxonomy that classifies methods from across the stack uniformly.
  arXiv  Detail & Related papers  (2026-07-09T01:15:03Z)
• What Should a Streaming Video Model Remember? [28.189301152079214]
  Streaming video understanding models must answer queries at any moment during an ongoing stream, using only what they have observed so far and under fixed memory and computation budgets.<n>We propose textbfSelectStream, a selective latent-memory framework that keeps the current observation directly visible to a frozen VLM while exposing historical information only through a compact, query-conditioned evidence budget.<n> Experimental results show that SelectStream achieves strong online streaming performance and preserves general video understanding, reaching 82.67% on StreamingBench, 67.03% on OVO-Bench, and 74.4% average accuracy on offline video benchmarks
  arXiv  Detail & Related papers  (2026-06-15T07:50:19Z)
• Fast Unlearning at Scale via Margin Self-Correction [52.46927918952516]
  Language-model unlearning updates a trained model to behave as if it had not seen selected training examples.<n>MASC achieves a competitive forget-retain trade-off at a fraction of the computational cost of existing baselines.
  arXiv  Detail & Related papers  (2026-06-01T21:49:54Z)
• Don't Pause! Every prediction matters in a streaming video [55.509551643600794]
  We introduce SPOT-Bench, featuring multi-turn proactive queries that evaluate general streaming perception and assistive capabilities.<n> SPOT-Bench comes with Timeliness-F1, a metric that measures streaming predictions by their temporal precision and balanced coverage across the entire video.<n>Our benchmark reveals: (i) offline models detect events reliably but spam predictions unprompted; (ii) post-training for silence reduces spamming but induces unresponsiveness; (iii) half of the streaming video expects no response, which we term dead-time - compute spent here does not affect response latency.
  arXiv  Detail & Related papers  (2026-04-27T11:07:03Z)
• Gated KalmaNet: A Fading Memory Layer Through Test-Time Ridge Regression [53.48692193399171]
  Gated KalmaNet (GKA) is a layer that reduces the gap by accounting for the full past when predicting the next token.<n>We solve an online ridge regression problem at test time, with constant memory and linear compute cost in the sequence length.<n>On long-context, GKA excels at real-world RAG and LongQA tasks up to 128k tokens, achieving more than $10$% relative improvement over other fading memory baselines.
  arXiv  Detail & Related papers  (2025-11-26T03:26:37Z)
• Rethinking Metrics and Benchmarks of Video Anomaly Detection [58.37571339811799]
  Video Anomaly Detection (VAD) aims to detect anomalies that deviate from expectation.<n>Existing VAD metrics are influenced by single annotation bias.<n>Existing benchmarks lack the capability to evaluate scene overfitting of fully/weakly-supervised algorithms.
  arXiv  Detail & Related papers  (2025-05-25T08:09:42Z)
• Simple linear attention language models balance the recall-throughput tradeoff [60.06020449520365]
  We propose BASED, a simple architecture combining linear and sliding window attention.<n>We train language models up to 1.3b parameters and show that BASED matches the strongest sub-quadratic models in perplexity and outperforms them on real-world recall-intensive tasks by 6.22 accuracy points.
  arXiv  Detail & Related papers  (2024-02-28T19:28:27Z)
• Streaming Anomaly Detection [7.60882697435906]
  We first propose MIDAS which uses a count-min sketch to detect anomalous edges in dynamic graphs in an online manner. We then extend the count-min sketch to a Higher-Order sketch to capture complex relations in graph data. Using this sketch, we propose four streaming methods to detect edge and subgraph anomalies.
  arXiv  Detail & Related papers  (2023-01-30T18:59:51Z)
• Real-Time Anomaly Detection in Edge Streams [49.26098240310257]
  We propose MIDAS, which focuses on detecting microcluster anomalies, or suddenly arriving groups of suspiciously similar edges. We further propose MIDAS-F, to solve the problem by which anomalies are incorporated into the algorithm's internal states. Experiments show that MIDAS-F has significantly higher accuracy than MIDAS.
  arXiv  Detail & Related papers  (2020-09-17T17:59:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.