V-CAST: Video Curvature-Aware Spatio-Temporal Pruning for Efficient Video Large Language Models

• URL: http://arxiv.org/abs/2603.27650v1
• Date: Sun, 29 Mar 2026 11:53:32 GMT
• Title: V-CAST: Video Curvature-Aware Spatio-Temporal Pruning for Efficient Video Large Language Models
• Abstract summary: Video language models (VideoLLMs) show strong capability in understanding, yet long-context inference is still dominated by massive redundant visual tokens in the prefill stage.<n>We propose V-CAST, a training-free, plug-and-play pruning policy for long-context video inference.
• Score: 48.80617385008755
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video large language models (VideoLLMs) show strong capability in video understanding, yet long-context inference is still dominated by massive redundant visual tokens in the prefill stage. We revisit token compression for VideoLLMs under a tight budget and identify a key bottleneck, namely insufficient spatio-temporal information coverage. Existing methods often introduce discontinuous coverage through coarse per-frame allocation or scene segmentation, and token merging can further misalign spatio-temporal coordinates under MRoPE-style discrete (t,h,w) bindings. To address these issues, we propose V-CAST (Video Curvature-Aware Spatio-Temporal Pruning), a training-free, plug-and-play pruning policy for long-context video inference. V-CAST casts token compression as a trajectory approximation problem and introduces a curvature-guided temporal allocation module that routes per-frame token budgets to semantic turns and event boundaries. It further adopts a dual-anchor spatial selection mechanism that preserves high-entropy visual evidence without attention intervention, while keeping retained tokens at their original coordinates to maintain positional alignment. Extensive experiments across multiple VideoLLMs of different architectures and scales demonstrate that V-CAST achieves 98.6% of the original performance, outperforms the second-best method by +1.1% on average, and reduces peak memory and total latency to 86.7% and 86.4% of vanilla Qwen3-VL-8B-Instruct.

Related papers

• DynaTok: Temporally Adaptive and Positional Bias-Aware Token Compression for Video-LLMs [2.6035240359441274]
  DynaTok is a training-free, temporally adaptive and bias-aware token compression framework.<n>It allocates token budgets across both temporal and spatial dimensions.<n>It retains over 95% of baseline accuracy even with a 90% token reduction.
  arXiv  Detail & Related papers  (2026-05-19T04:02:01Z)
• TV-RAG: A Temporal-aware and Semantic Entropy-Weighted Framework for Long Video Retrieval and Understanding [14.570869250170139]
  TV-RAG is a training-free architecture that couples temporal alignment with entropy-guided semantics to improve long-video reasoning.<n>By weaving these temporal and semantic signals together, TV-RAG realises a dual-level reasoning routine that can be grafted onto any LVLM without re-training or fine-tuning.
  arXiv  Detail & Related papers  (2025-12-29T14:10:22Z)
• State Space Prompting via Gathering and Spreading Spatio-Temporal Information for Video Understanding [50.866929044215965]
  We propose a State Space Prompting (SSP) method for video understanding.<n>SSP combines intra-frame inter-frame prompts to aggregate and propagate keytemporal information in the video.<n>Our SSP significantly outperforms existing SOTA methods by 2.76% on average.
  arXiv  Detail & Related papers  (2025-10-14T05:30:36Z)
• Towards Robust and Generalizable Continuous Space-Time Video Super-Resolution with Events [71.2439653098351]
  Continuous space-time video super-STVSR has garnered increasing interest for its capability to reconstruct high-resolution and high-frame-rate videos at arbitrary temporal scales.<n>We present EvEnhancer, a novel approach that marries unique properties of high temporal and high dynamic range encapsulated in event streams.<n>Our method achieves state-of-the-art performance on both synthetic and real-world datasets, while maintaining generalizability at OOD scales.
  arXiv  Detail & Related papers  (2025-10-04T15:23:07Z)
• Dense Video Understanding with Gated Residual Tokenization [49.17263029080152]
  High temporal resolution is essential for capturing fine-grained details in video understanding.<n>Current benchmarks rely mostly on low-frame-rate sampling.<n>Dense Video Understanding (DVU) enables high-FPS video comprehension by reducing both tokenization time and token overhead.
  arXiv  Detail & Related papers  (2025-09-17T17:34:40Z)
• Temporal Cluster Assignment for Efficient Real-Time Video Segmentation [9.248291541710781]
  Vision Transformers have substantially advanced the capabilities of segmentation models across both image and video domains.<n>The window-based attention mechanism of Swin requires a fixed number of tokens per window, limiting the applicability of conventional pruning techniques.<n>We introduce Temporal Cluster Assignment (TCA), a lightweight and effective, fine-tuning-free strategy that enhances token clustering by leveraging temporal coherence.
  arXiv  Detail & Related papers  (2025-08-07T20:52:49Z)
• Exploiting Temporal State Space Sharing for Video Semantic Segmentation [53.8810901249897]
  Video semantic segmentation (VSS) plays a vital role in understanding the temporal evolution of scenes.<n>Traditional methods often segment videos frame-by-frame or in a short temporal window, leading to limited temporal context, redundant computations, and heavy memory requirements.<n>We introduce a Temporal Video State Space Sharing architecture to leverage Mamba state space models for temporal feature sharing.<n>Our model features a selective gating mechanism that efficiently propagates relevant information across video frames, eliminating the need for a memory-heavy feature pool.
  arXiv  Detail & Related papers  (2025-03-26T01:47:42Z)
• Implicit Temporal Modeling with Learnable Alignment for Video Recognition [95.82093301212964]
  We propose a novel Implicit Learnable Alignment (ILA) method, which minimizes the temporal modeling effort while achieving incredibly high performance. ILA achieves a top-1 accuracy of 88.7% on Kinetics-400 with much fewer FLOPs compared with Swin-L and ViViT-H.
  arXiv  Detail & Related papers  (2023-04-20T17:11:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.