GSTEP: Global Spatio-Temporal Density-Driven Visual Token Pruning for Efficient Video Large Language Models

• URL: http://arxiv.org/abs/2608.03083v1
• Date: Tue, 04 Aug 2026 03:51:46 GMT
• Title: GSTEP: Global Spatio-Temporal Density-Driven Visual Token Pruning for Efficient Video Large Language Models
• Abstract summary: We propose GSTEP, a plug-and-play pruning framework that models video as a continuous spatial-temporal information flow.<n>GSTEP consistently achieves strong accuracy-efficiency trade-offs and generalizes well across model architectures and evaluation settings.
• Score: 23.830073542329913
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video large language models (VideoLLMs) achieve strong video understanding performance, but their inference remains expensive due to the large number of redundant spatio-temporal visual tokens in long videos. Existing token pruning methods alleviate this cost by reducing redundant tokens, yet most of them rely on segment-level local pruning, where videos are partitioned into isolated segments and tokens are selected independently within each segment. Such designs may under-preserve short but semantically dense segments and discard tokens that appear non-salient locally but remain critical from a global perspective. To address this issue, we propose GSTEP (Global Spatio-Temporal Density Pruning), a plug-and-play pruning framework that models video as a continuous spatio-temporal information flow. GSTEP constructs a token-level spatio-temporal density by combining a continuous temporal density, obtained from a smoothed centered frame-level change signal, with intra-frame spatial density, and then performs global token sampling by jointly balancing information density and coverage. Extensive experiments on multiple VideoLLMs and public benchmarks demonstrate that GSTEP consistently achieves strong accuracy-efficiency trade-offs and generalizes well across model architectures and evaluation settings. On LLaVA-OneVision-7B, GSTEP prunes 75% of visual tokens, preserves up to 100.2% of the original average performance across benchmarks, and achieves a 1.17 end-to-end speedup.

Related papers

• Tango: Taming Visual Signals for Efficient Video Large Language Models [51.43125826972642]
  This work revisits and advances the two predominant token-pruning paradigms: attention-based selection and similarity-based clustering.<n>To address these bottlenecks, we propose Tango, a novel framework designed to optimize the utilization of visual signals.
  arXiv  Detail & Related papers  (2026-04-10T17:59:56Z)
• HieraVid: Hierarchical Token Pruning for Fast Video Large Language Models [43.7747080569325]
  HieraVid is a hierarchical pruning framework that progressively and dynamically reduces visual redundancy.<n>We conduct extensive experiments on four widely used video understanding benchmarks to evaluate the effectiveness of HieraVid.<n>With only 30% of tokens retained, HieraVid achieves new state-of-the-art performance, while maintaining over 98% and 99% of the performance of LLaVA-Video-7B and LLaVA-OneVision-7B, respectively.
  arXiv  Detail & Related papers  (2026-04-02T10:40:15Z)
• Unified Spatio-Temporal Token Scoring for Efficient Video VLMs [61.08183446817756]
  Token pruning is essential for enhancing the computational efficiency of vision-language models.<n>We introduce Spatio-Temporal Token Scoring (STTS), a simple and lightweight module that prunes vision tokens across both the ViT and the LLM.<n>STTS prunes 50% of vision tokens throughout the entire architecture, resulting in a 62% improvement in efficiency during both training and inference.
  arXiv  Detail & Related papers  (2026-03-18T17:59:56Z)
• Token Reduction via Local and Global Contexts Optimization for Efficient Video Large Language Models [61.11154533305096]
  Video Large Language Models (VLLMs) demonstrate strong video understanding but suffer from inefficiency due to redundant visual tokens.<n>We propose a new perspective that elaborates token textbfAnchors within intra-frame and inter-frame contexts.<n>Our proposed AOT obtains competitive performances across various short- and long-video benchmarks on leading video LLMs.
  arXiv  Detail & Related papers  (2026-03-02T03:06:40Z)
• TrajTok: Learning Trajectory Tokens enables better Video Understanding [63.1260672430712]
  Tokenization in video models, typically through patchification, generates an excessive and redundant number of tokens.<n>We propose TrajTok, an end-to-end video tokenizer module that is fully integrated and co-trained with video models for a downstream objective.<n>We show that it can be seamlessly integrated as either a probing head for pretrained visual features (TrajAdapter) or an alignment connector in vision-language models (TrajVLM) with especially strong performance in long-video reasoning.
  arXiv  Detail & Related papers  (2026-02-26T09:15:34Z)
• Spatio-Temporal Attention for Consistent Video Semantic Segmentation in Automated Driving [0.46664938579243564]
  We propose apatio S-Temporal Attention (STA) mechanism that extends transformer attention blocks to incorporate multi-frame context.<n>Our approach modifies standard self-attention to process-temporal feature sequences while maintaining computational efficiency.<n>A comprehensive evaluation on the Cityscapes and BDD100k datasets shows substantial improvements of 9.20 percentage points in temporal consistency metrics.
  arXiv  Detail & Related papers  (2026-02-10T18:18:37Z)
• Fast SAM2 with Text-Driven Token Pruning [52.8350457627401]
  Segment Anything Model 2 (SAM2), a vision computation model has significantly advanced in prompt-driven video object segmentation.<n>SAM2 pipelines propagate all visual tokens produced by the image encoder through downstream temporal reasoning modules, regardless of their relevance to the target object.<n>We introduce a text-guided token pruning framework that improves inference efficiency by selectively reducing token density prior to temporal propagation.
  arXiv  Detail & Related papers  (2025-12-24T18:59:05Z)
• Dense Video Understanding with Gated Residual Tokenization [49.17263029080152]
  High temporal resolution is essential for capturing fine-grained details in video understanding.<n>Current benchmarks rely mostly on low-frame-rate sampling.<n>Dense Video Understanding (DVU) enables high-FPS video comprehension by reducing both tokenization time and token overhead.
  arXiv  Detail & Related papers  (2025-09-17T17:34:40Z)
• VQToken: Neural Discrete Token Representation Learning for Extreme Token Reduction in Video Large Language Models [35.38573641029626]
  We introduce the novel task of Extreme Short Token Reduction, which aims to represent entire videos using a minimal set of discrete tokens.<n>On the Extreme Short Token Reduction task, our VQToken compresses sequences to just 0.07 percent of their original length while incurring only a 0.66 percent drop in accuracy on the NextQA-MC benchmark.
  arXiv  Detail & Related papers  (2025-03-21T09:46:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.