FadeMem: Distance-Aware Memory Consolidation for Autoregressive Video Diffusion

• URL: http://arxiv.org/abs/2606.10671v1
• Date: Tue, 09 Jun 2026 10:22:18 GMT
• Title: FadeMem: Distance-Aware Memory Consolidation for Autoregressive Video Diffusion
• Abstract summary: FadeMem organizes historical KV blocks into a temporal hierarchy under a fixed cache budget.<n>New history is inserted as fine-grained entries, while older adjacent entries are progressively merged.<n>Experiments show improved subject consistency, background stability, and temporal coherence over existing bounded-cache strategies.
• Score: 59.207505503284715
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autoregressive video generators synthesize long videos by generating successive temporal segments, but their historical KV cache grows with video length. Existing bounded-cache methods reduce this cost with local windows, sink tokens, or compressed memory states, yet they usually assign fixed roles to different parts of the history. We propose FadeMem, a distance-aware KV memory consolidation mechanism that organizes historical KV blocks into a temporal hierarchy under a fixed cache budget. This design is motivated by frequency-dependent temporal decay: fine details decorrelate quickly, while coarse scene structure and identity remain useful over longer horizons. During generation, new history is inserted as fine-grained entries, while older adjacent entries are progressively merged under a power-law temporal allocation schedule, yielding a dense-near, sparse-far memory within one cache. Without architectural changes, FadeMem preserves recent context for short-term dynamics and compact long-range anchors for identity and scene coherence. Experiments show improved subject consistency, background stability, and temporal coherence over existing bounded-cache strategies.

Related papers

• LayerRecall: A State-Conditioned Memory Router for Long-Horizon Consistency in Video Generation [22.878882105342182]
  Autoregressive video diffusion produces chunks from a bounded recent context.<n>Recency-based caching evicts historical cues needed when subjects, objects, scenes, or attributes reappear.<n>LayerRecall retrieves relevant historical K/V states and injects them only into backbone-specific memory-sensitive layers.
  arXiv  Detail & Related papers  (2026-08-28T15:49:13Z)
• TetherCache: Stabilizing Autoregressive Long-Form Video Generation with Gated Recall and Trusted Alignment [51.33418612284208]
  We propose TetherCache, a training-free and plug-and-play cache management strategy for drift-resistant long video generation.<n>Gated Recall with Attention-Diversity Balancing selects long-range memory frames using a gated score.<n>TAME lightly edits newly recalled memory tokens by aligning their statistics to a trusted context distribution.
  arXiv  Detail & Related papers  (2026-06-11T08:16:08Z)
• SlotMemory: Object-Centric KV Memory for Streaming Long-Video Generation [30.157534925157844]
  SlotMemory is an object-centric Key-Value memory mechanism for streaming video diffusion.<n>Our approach shifts the memory abstraction from "when" an event occurred to "what" is being represented by decomposing the transformer's key-value manifold.
  arXiv  Detail & Related papers  (2026-05-29T09:04:26Z)
• DySink: Dynamic Frame Sinks for Autoregressive Long Video Generation [51.58559206569209]
  We propose DySink, a retrieval-based framework that maintains a compact memory bank and selects visually relevant historical frames as dynamic frame sinks.<n> Experiments on minute-long videos show that DySink consistently improves dynamic degree over strong baselines while also achieving higher temporal quality.
  arXiv  Detail & Related papers  (2026-05-20T11:01:01Z)
• Attend Locally, Remember Linearly: Linear Attention as Cross-Frame Memory for Autoregressive Video Diffusion [61.57938553036056]
  ARL2 is a hybrid attention module that replaces quadratic cross-frame attention with a fixed-size recurrent state.<n>We decompose self-attention into two branches: an intra-frame softmax branch for spatial detail and local dependencies, and an inter-frame recurrent linear branch that maintains a fixed-size state for streaming context.<n>With 75% of layers replaced by hybrid linear attention, the model achieves up to 2.26 wall-clock speedup and 54% memory reduction, while maintaining comparable quality with improving temporal consistency.
  arXiv  Detail & Related papers  (2026-05-15T19:33:45Z)
• Echo-Forcing: A Scene Memory Framework for Interactive Long Video Generation [48.476317015122625]
  Echo-Forcing is a training-free scene memory framework for interactive long video generation.<n>It supports smooth transitions, hard cuts, and long-range scene recall under a bounded cache budget.
  arXiv  Detail & Related papers  (2026-05-15T14:33:09Z)
• SWIFT: Prompt-Adaptive Memory for Efficient Interactive Long Video Generation [31.105007908298003]
  Streaming long-video generation faces a central challenge in continuous semantic switching.<n>Current approaches rely on cache rebuilding at prompt boundaries or fixed memory budgets.<n>We present SWIFT, a training-free framework for multi-prompt long-video generation.
  arXiv  Detail & Related papers  (2026-05-10T09:37:56Z)
• MemRoPE: Training-Free Infinite Video Generation via Evolving Memory Tokens [34.228121359393775]
  We introduce MemRoPE, a training-free framework with two co-designed components.<n>Online RoPE Indexing caches unrotated keys and applies positional embeddings dynamically at attention time.<n>MemRoPE outperforms existing methods in temporal coherence, visual fidelity, and subject consistency across minute- to hour-scale generation.
  arXiv  Detail & Related papers  (2026-03-12T23:14:16Z)
• PackCache: A Training-Free Acceleration Method for Unified Autoregressive Video Generation via Compact KV-Cache [61.57938553036056]
  We introduce PackCache, a training-free KV-cache management method that compacts the KV cache through three coordinated mechanisms.<n>In terms of efficiency, PackCache accelerates end-to-end generation by 1.7-2.2x on 48-frame long sequences.
  arXiv  Detail & Related papers  (2026-01-07T19:51:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.