Decouple and Cache: KV Cache Construction for Streaming Video Understanding

• URL: http://arxiv.org/abs/2605.01858v1
• Date: Sun, 03 May 2026 13:02:44 GMT
• Title: Decouple and Cache: KV Cache Construction for Streaming Video Understanding
• Abstract summary: Streaming video understanding requires processing unbounded video streams with limited memory and computation.<n>We propose Decoupled Streaming Cache(DSCache), a training-free cache construction mechanism that adapts pretrained offline models to streaming settings.<n> Experiments on Streaming Video QA benchmarks demonstrate DSCache's state-of-the-art performance, with an average 2.5% accuracy gains over prior methods.
• Score: 53.55135022958052
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Streaming video understanding requires processing unbounded video streams with limited memory and computation, posing two key challenges. First, continuously constructing new and evicting old key-value(KV) caches is required for unbounded streams. Secondly, due to the high cost of collecting and training on unbounded streams, models must learn from short sequences while generalizing to long streams. Existing streaming VideoVLLMs fail to scale to unbounded video streams or focus on cache reuse strategies, leaving the impact of cache construction underexplored. In this paper, we propose Decoupled Streaming Cache(DSCache), a training-free cache construction mechanism that adapts pretrained offline models to streaming settings. DSCache maintains a cumulative past KV cache while constructing a separate instant cache on-demand, decoupled from past caches to preserve the informativeness of recent inputs. To enable position extrapolation beyond the training length, DSCache further incorporates a position-agnostic encoding strategy, ensuring KV caches to support unseen positions and preventing position overflow. Experiments on Streaming Video QA benchmarks demonstrate DSCache's state-of-the-art performance, with an average 2.5% accuracy gains over prior methods.

Related papers

• VisCache: Visual KV Cache Pruning for Efficient Vision Large Language Model Inference [17.932004205808486]
  VisCache is a framework for coarse-to-fine textbfVisual KV textbfCache pruning without training.<n>PruneKV is a surgical KV compression algorithm tailored to the attention dynamics of Vision Large Language Models.<n>VisCache consistently outperforms existing baselines.
  arXiv  Detail & Related papers  (2026-08-25T04:52:05Z)
• PreciseCache: Precise Feature Caching for Efficient and High-fidelity Video Generation [35.47114707080758]
  High computational costs and slow inference hinder the practical application of video generation models.<n>We propose textbfPreciseCache, a plug-and-play framework that precisely detects and skips truly redundant computations.
  arXiv  Detail & Related papers  (2026-03-01T08:08:49Z)
• Flow caching for autoregressive video generation [72.10021661412364]
  We present FlowCache, the first caching framework specifically designed for autoregressive video generation.<n>Our method achieves remarkable speedups of 2.38 times on MAGI-1 and 6.7 times on SkyReels-V2, with negligible quality degradation.
  arXiv  Detail & Related papers  (2026-02-11T13:11:04Z)
• PackCache: A Training-Free Acceleration Method for Unified Autoregressive Video Generation via Compact KV-Cache [61.57938553036056]
  We introduce PackCache, a training-free KV-cache management method that compacts the KV cache through three coordinated mechanisms.<n>In terms of efficiency, PackCache accelerates end-to-end generation by 1.7-2.2x on 48-frame long sequences.
  arXiv  Detail & Related papers  (2026-01-07T19:51:06Z)
• StreamKV: Streaming Video Question-Answering with Segment-based KV Cache Retrieval and Compression [95.59657871147846]
  We propose textbfStreamKV, a framework that seamlessly equips Video-LLMs with advanced KV cache retrieval and compression.<n>Experiments on public StreamingVQA benchmarks demonstrate that StreamKV significantly outperforms existing Online Video-LLMs.
  arXiv  Detail & Related papers  (2025-11-10T16:25:03Z)
• dKV-Cache: The Cache for Diffusion Language Models [53.85291644298835]
  Diffusion Language Models (DLMs) have been seen as a promising competitor for autoregressive language models.<n>We propose a KV-cache-like mechanism, delayed KV-Cache, for the denoising process of DLMs.<n>Our approach is motivated by the observation that different tokens have distinct representation dynamics throughout the diffusion process.
  arXiv  Detail & Related papers  (2025-05-21T17:32:10Z)
• FasterCache: Training-Free Video Diffusion Model Acceleration with High Quality [58.80996741843102]
  FasterCache is a training-free strategy designed to accelerate the inference of video diffusion models with high-quality generation.<n>We show that FasterCache can significantly accelerate video generation while keeping video quality comparable to the baseline.
  arXiv  Detail & Related papers  (2024-10-25T07:24:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.