Separating Stream Stability from Long-Term Recall in Language Models

• URL: http://arxiv.org/abs/2609.07282v1
• Date: Mon, 07 Sep 2026 09:49:32 GMT
• Title: Separating Stream Stability from Long-Term Recall in Language Models
• Abstract summary: An attention sink can stabilize autoregressive generation over an indefinitely long stream while the model remains unable to use content that has left its recent-token cache.<n>We introduce three horizons: the stability horizon, over which predictive behavior remains well behaved; the access horizon, over which past content can still causally affect the output; and the utility horizon, over which a task retains acceptable performance.<n>Applying the framework to attention-sink streaming clarifies its strength, constant-memory, stable generation, without treating anchor tokens as semantic memory.
• Score: 14.319140213074045
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Methods for streaming language models are often discussed alongside long-context and memory systems, although they solve different problems. An attention sink can stabilize autoregressive generation over an indefinitely long stream while the model remains unable to use content that has left its recent-token cache. We argue that this distinction should be explicit in system claims and evaluation. We introduce three horizons: the stability horizon, over which predictive behavior remains well behaved; the access horizon, over which past content can still causally affect the output; and the utility horizon, over which a task retains acceptable performance. We show constructively that the stability horizon can be infinite while the access and utility horizons are finite. We then propose ThreeH, an evaluation contract that measures all three horizons under a common state and compute budget. Applying the framework to attention-sink streaming clarifies its strength, constant-memory, stable generation, without treating anchor tokens as semantic memory. The framework exposes roles for cache policies, recurrent state, retrieval, and external memory. Experiments on 128K-token streams, delayed binding recall, and delayed decisions show that attention sinks preserve local modeling but not content beyond the active cache; recurrent and retrieval state extend the semantic horizon.

Related papers

• Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understanding [64.34948094316685]
  We introduce LatentStream, a progressive latent working memory framework that shifts streaming memory from store-and-retrieve to retrieve-and-internalize.<n>LatentStream achieves new state-of-the-art results on existing online and offline video benchmarks.
  arXiv  Detail & Related papers  (2026-09-03T17:28:14Z)
• LayerRecall: A State-Conditioned Memory Router for Long-Horizon Consistency in Video Generation [22.878882105342182]
  Autoregressive video diffusion produces chunks from a bounded recent context.<n>Recency-based caching evicts historical cues needed when subjects, objects, scenes, or attributes reappear.<n>LayerRecall retrieves relevant historical K/V states and injects them only into backbone-specific memory-sensitive layers.
  arXiv  Detail & Related papers  (2026-08-28T15:49:13Z)
• TaoMate: Anchor-Guided Memory Bridging Evolving and Reference States for Real-Time Audio-Video Digital Human Generation [17.429355392513354]
  Real-time long-form digital-human generation relies on causal models to extend audio-visual content.<n>method is an anchor-guided persistent-memory framework for few-step joint audio-video generation.
  arXiv  Detail & Related papers  (2026-07-27T12:35:32Z)
• LiveStarPro: Proactive Streaming Video Understanding with Hierarchical Memory for Long-Horizon Streams [59.485485426790966]
  We introduce LiveStarPro, a live streaming assistant that is designed for proactive video understanding over long-horizon streams.<n>LiveStarPro consistently surpasses existing methods, attaining a 28.9% improvement in semantic correctness.<n>Its streaming key-value cache further yields a 1.58x inference speedup over the same model without caching.
  arXiv  Detail & Related papers  (2026-06-16T11:18:05Z)
• FadeMem: Distance-Aware Memory Consolidation for Autoregressive Video Diffusion [59.207505503284715]
  FadeMem organizes historical KV blocks into a temporal hierarchy under a fixed cache budget.<n>New history is inserted as fine-grained entries, while older adjacent entries are progressively merged.<n>Experiments show improved subject consistency, background stability, and temporal coherence over existing bounded-cache strategies.
  arXiv  Detail & Related papers  (2026-06-09T10:22:18Z)
• DySink: Dynamic Frame Sinks for Autoregressive Long Video Generation [51.58559206569209]
  We propose DySink, a retrieval-based framework that maintains a compact memory bank and selects visually relevant historical frames as dynamic frame sinks.<n> Experiments on minute-long videos show that DySink consistently improves dynamic degree over strong baselines while also achieving higher temporal quality.
  arXiv  Detail & Related papers  (2026-05-20T11:01:01Z)
• Relax Forcing: Relaxed KV-Memory for Consistent Long Video Generation [73.84423888025171]
  Autoregressive (AR) video diffusion has recently emerged as a promising paradigm for long video generation.<n>We show that extending generation to minute-scale horizons remains challenging due to progressive temporal degradation.<n>We introduce Relax Forcing, a structured temporal memory mechanism for AR diffusion.
  arXiv  Detail & Related papers  (2026-03-22T18:59:24Z)
• FILT3R: Latent State Adaptive Kalman Filter for Streaming 3D Reconstruction [51.56484100374058]
  Streaming 3D reconstruction maintains a persistent latent state that is updated online from incoming frames.<n>FILT3R casts recurrent state updates as state estimation in token space.<n>Code will be released at https://github.com/jinotter3/FILT3R.
  arXiv  Detail & Related papers  (2026-03-19T04:56:36Z)
• Bounded State in an Infinite Horizon: Proactive Hierarchical Memory for Ad-Hoc Recall over Streaming Dialogues [27.69314585543646]
  We present textbfProStream, a proactive hierarchical memory framework for streaming dialogues.<n>It enables ad-hoc memory recall on demand by reasoning over continuous streams with multi-granular distillation.<n>Experiments show that ProStream outperforms baselines in both accuracy and efficiency.
  arXiv  Detail & Related papers  (2026-03-05T07:25:25Z)
• OccSTeP: Benchmarking 4D Occupancy Spatio-Temporal Persistence [19.81329748424203]
  We introduce a new concept of 4D Occupancy Spatio-Temporal Persistence (OccSTeP)<n>This concept aims to address two tasks: (1) reactive forecasting: ''what happen'' and (2) proactive forecasting: "what would happen given a specific future action"<n>We propose OccSTeP-WM, a tokenizer-free world model that maintains a dense voxel-based scene state and fuses incrementally-temporal context over time.
  arXiv  Detail & Related papers  (2025-12-17T17:29:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.