EarlyTom: Early Token Compression Completes Fast Video Understanding

• URL: http://arxiv.org/abs/2605.30010v1
• Date: Thu, 28 May 2026 14:36:44 GMT
• Title: EarlyTom: Early Token Compression Completes Fast Video Understanding
• Abstract summary: Video large language models (Video-LLMs) have demonstrated strong capabilities in video understanding tasks.<n>Recent approaches achieve extremely low token retention ratios while maintaining accuracy comparable to full-token baselines.<n>We propose EarlyTom, a training-free token compression framework that performs early-stage visual token compression inside the vision encoder.
• Score: 18.40441750844911
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video large language models (Video-LLMs) have demonstrated strong capabilities in video understanding tasks. However, their practical deployment is still hindered by the inefficiency introduced by processing massive amounts of visual tokens. Although recent approaches achieve extremely low token retention ratios while maintaining accuracy comparable to full-token baselines, most of them perform compression only at the late stage of prefilling, leaving the efficiency of the vision encoder unoptimized. In this paper, we first show that vision encoding contributes a large portion to the time-to-first-token (TTFT). Therefore, instead of compressing visual tokens only after the vision encoder, performing compression inside the encoder still leaves substantial room for exploration. Based on this insight, we propose EarlyTom, a training-free token compression framework that performs early-stage visual token compression inside the vision encoder, enabling significantly better TTFT reduction and higher throughput. In addition, we introduce a decoupled spatial token selection strategy that improves the overall compression effectiveness. EarlyTom reduces TTFT by up to 2.65x and FLOPs by up to 61% on a single NVIDIA A100 GPU for the LLaVA-OneVision-7B model, while maintaining accuracy comparable to the full-token baseline. These improvements substantially enhance the practicality of deploying Video-LLMs in real-world production scenarios.

Related papers

• VisCo: Leveraging Large Language Models as Intrinsic Encoders for Visual Token Compression [52.23010472235226]
  Vision-language models (VLMs) process large numbers of visual tokens, resulting in substantial inference and overhead latency.<n>We propose VisCo, a training-efficient self-compression framework that reuses the pretrained VLM itself as an intrinsic compressor.
  arXiv  Detail & Related papers  (2026-07-14T13:21:36Z)
• UIPress: Bringing Optical Token Compression to UI-to-Code Generation [2.462537008029033]
  We propose a lightweight learned compression module for UI-to-Code generation.<n> UIPress combines depthwise-separable convolutions, element-guided spatial reweighting, and Transformer refinement to compress visual tokens to a fixed budget of 256.<n>To the best of our knowledge, UIPress is the first encoder-side learned compression method for the UI-to-Code task.
  arXiv  Detail & Related papers  (2026-04-10T15:58:31Z)
• Unified Spatio-Temporal Token Scoring for Efficient Video VLMs [61.08183446817756]
  Token pruning is essential for enhancing the computational efficiency of vision-language models.<n>We introduce Spatio-Temporal Token Scoring (STTS), a simple and lightweight module that prunes vision tokens across both the ViT and the LLM.<n>STTS prunes 50% of vision tokens throughout the entire architecture, resulting in a 62% improvement in efficiency during both training and inference.
  arXiv  Detail & Related papers  (2026-03-18T17:59:56Z)
• ProGIC: Progressive and Lightweight Generative Image Compression with Residual Vector Quantization [59.481950697968706]
  We propose Progressive Generative Image Compression (ProGIC), a compact built on residual vector quantization (RVQ)<n>In RVQ, a sequence of vector quantizers encodes the residuals stage by stage, each with its own codebook.<n>We pair this with a lightweight backbone based on depthwise-separable convolutions and small attention blocks, enabling practical deployment on both GPU and CPU-only devices.
  arXiv  Detail & Related papers  (2026-03-03T11:47:05Z)
• PIO-FVLM: Rethinking Training-Free Visual Token Reduction for VLM Acceleration from an Inference-Objective Perspective [59.24570811503256]
  We propose PIO-FVLM to reduce redundant visual tokens in vision-models (VLMs) to accelerate inference.<n>The proposed PIO-FVLM is training-free and compatible with FlashAttention, friendly to practical application and deployment.<n>On LLaVA-Next-7B, PIO-FVLM retains just 11.1% of visual tokens but maintains 97.2% of the original performance.
  arXiv  Detail & Related papers  (2026-02-04T15:33:10Z)
• METok: Multi-Stage Event-based Token Compression for Efficient Long Video Understanding [55.38256656122857]
  We propose METok, a training-free, Multi-stage Event-based Token compression framework.<n>We show METok achieves an optimal trade-off between efficiency and accuracy by dynamically selecting informative visual tokens.<n>For instance, equipping LongVA-7B with METok realizes an 80.6% FLOPs reduction and 93.5% KV Cache memory savings.
  arXiv  Detail & Related papers  (2025-06-03T13:19:41Z)
• Vision-centric Token Compression in Large Language Model [51.92055188780033]
  Vision Centric Token Compression (Vist) is a slow-fast compression framework that mirrors human reading.<n>On eleven in-context learning benchmarks, Vist achieves the same accuracy with 2.3 times fewer tokens, cutting FLOPs by 16% and memory by 50%.
  arXiv  Detail & Related papers  (2025-02-02T13:10:06Z)
• DyCoke: Dynamic Compression of Tokens for Fast Video Large Language Models [28.379533608574814]
  We present DyCoke, a training-free token compression method to optimize token representation and accelerate video large language models.<n>DyCoke incorporates a plug-and-play temporal compression module to minimize temporal redundancy by merging redundant tokens across frames.<n>It ensures high-quality inference by dynamically retaining the critical tokens at each decoding step.
  arXiv  Detail & Related papers  (2024-11-22T15:55:19Z)
• VoCo-LLaMA: Towards Vision Compression with Large Language Models [31.398537194299752]
  Vision-Language Models (VLMs) have achieved remarkable success in various multi-modal tasks, but they are often bottlenecked by the limited context window.<n>We propose VoCo-LLaMA, the first approach to compress vision tokens using LLMs.<n>Our method achieves minimal performance loss with a compression ratio of 576$times$, resulting in up to 94.8$%$ fewer FLOPs and 69.6$%$ acceleration in inference time.
  arXiv  Detail & Related papers  (2024-06-18T05:05:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.