CRAFT: Compression via Recursive Adaptive Fusion of Video Tokens for Vision-Language Models

• URL: http://arxiv.org/abs/2608.01644v1
• Date: Mon, 03 Aug 2026 03:27:46 GMT
• Title: CRAFT: Compression via Recursive Adaptive Fusion of Video Tokens for Vision-Language Models
• Abstract summary: We propose CRAFT: Compression via Resive Adaptive Fusion Video Tokens.<n>CRAFT consistently outperforms prior state-of-the-art token-compression methods.<n>At about $8times$ compression, it retains roughly $97% of the backbone's average accuracy.
• Score: 9.660619113725843
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In video understanding, vision-language models (VLMs) must ingest massive numbers of visual tokens, causing the computational and memory cost of the prefill stage to rise sharply. Such visual sequences are highly redundant along the spatio-temporal dimension, yet a high compression ratio is often accompanied by the loss of critical details. Existing token-compression methods either employ heuristic, training-free compression with limited content adaptivity or introduce additional modules that require expensive alignment training, leaving the trade-off between efficiency and adaptivity unresolved. To alleviate this limitation, we propose CRAFT: Compression via Recursive Adaptive Fusion of Video Tokens. CRAFT recursively merges tokens by decoupling parameter-free token selection from learnable token fusion: global similarity determines which tokens to merge, while a position-aware weighting module and a content-adaptive channel-wise gate learn how to fuse them. The whole compression pipeline is query-agnostic. Because every retained token is a linear combination of the original tokens, CRAFT preserves their true spatio-temporal coordinates and stays aligned with the pre-trained language model's input distribution. Experiments on multiple representative video benchmarks show that CRAFT consistently outperforms prior state-of-the-art token-compression methods. At about $8\times$ compression, it retains roughly $97\%$ of the backbone's average accuracy and shows significant efficiency improvement.

Related papers

• Keep-or-Drop? Adaptive Tokenizer for Compact Video Representation [3.7696381512190804]
  We present an adaptive token selector that discards uninformative tokens.<n>We empirically show that our model achieves strong reconstruction and generation quality at a state-of-the-art compression ratio.
  arXiv  Detail & Related papers  (2026-08-25T09:19:22Z)
• EVATok: Adaptive Length Video Tokenization for Efficient Visual Autoregressive Generation [80.13014959623452]
  EVATok is a framework to produce $textbfE$fficient $textbfV$ideo $textbfA$daptive $textbfTok$enizers.<n>We show that EVATok achieves superior reconstruction and state-of-the-art class-to-video generation on UCF-101.
  arXiv  Detail & Related papers  (2026-03-12T17:59:59Z)
• ConsensusDrop: Fusing Visual and Cross-Modal Saliency for Efficient Vision Language Models [4.273730624882391]
  Vision-Language Models (VLMs) are expensive because the LLM processes hundreds of largely redundant visual tokens.<n>We show that neither signal alone is sufficient: fusing them consistently improves performance compared to unimodal visual token selection (ranking)<n>We propose textbfConsensusDrop, a training-free framework that derives a emphconsensus ranking by reconciling vision encoder saliency with query-aware cross-attention.
  arXiv  Detail & Related papers  (2026-02-01T00:28:55Z)
• InfoTok: Adaptive Discrete Video Tokenizer via Information-Theoretic Compression [114.03378443007074]
  Current tokenizers rigidly compress all content at a fixed rate, leading to redundancy or information loss.<n>This paper introduces InfoTok, a principled framework for adaptive video tokenization.<n>We develop a transformer-based adaptive compressor that enables adaptive tokenization.
  arXiv  Detail & Related papers  (2025-12-18T17:13:59Z)
• VisionSelector: End-to-End Learnable Visual Token Compression for Efficient Multimodal LLMs [82.72388893596555]
  Multimodal Large Language Models (MLLMs) encounter significant computational and memory bottlenecks.<n>Previous token compression techniques are often constrained by rules that risk discarding critical information.<n>We reformulate token compression as a lightweight plug-and-play framework that reformulates token compression into an end-to-end learnable decision process.
  arXiv  Detail & Related papers  (2025-10-18T17:54:18Z)
• Learning Adaptive and Temporally Causal Video Tokenization in a 1D Latent Space [94.07013629356113]
  AdapTok is an adaptive temporal causal video tokenizer that can flexibly allocate tokens for different frames based on video content.<n>AdapTok consistently improves reconstruction quality and generation performance under different token budgets.
  arXiv  Detail & Related papers  (2025-05-22T17:59:02Z)
• VQToken: Neural Discrete Token Representation Learning for Extreme Token Reduction in Video Large Language Models [35.38573641029626]
  We introduce the novel task of Extreme Short Token Reduction, which aims to represent entire videos using a minimal set of discrete tokens.<n>On the Extreme Short Token Reduction task, our VQToken compresses sequences to just 0.07 percent of their original length while incurring only a 0.66 percent drop in accuracy on the NextQA-MC benchmark.
  arXiv  Detail & Related papers  (2025-03-21T09:46:31Z)
• Video Token Merging for Long-form Video Understanding [17.59960070514554]
  We propose a learnable video token merging algorithm that dynamically merges tokens based on their saliency. Our approach significantly reduces memory costs by 84% and boosts throughput by approximately 6.89 times compared to baseline algorithms.
  arXiv  Detail & Related papers  (2024-10-31T09:55:32Z)
• ElasticTok: Adaptive Tokenization for Image and Video [109.75935878130582]
  We introduce ElasticTok, a method that conditions on prior frames to adaptively encode a frame into a variable number of tokens.<n>During inference, ElasticTok can dynamically allocate tokens when needed.<n>Our evaluations on images and video demonstrate the effectiveness of our approach in efficient token usage.
  arXiv  Detail & Related papers  (2024-10-10T20:54:15Z)
• Token-level Correlation-guided Compression for Efficient Multimodal Document Understanding [54.532578213126065]
  Most document understanding methods preserve all tokens within sub-images and treat them equally. This neglects their different informativeness and leads to a significant increase in the number of image tokens. We propose Token-level Correlation-guided Compression, a parameter-free and plug-and-play methodology to optimize token processing.
  arXiv  Detail & Related papers  (2024-07-19T16:11:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.