Tango: Taming Visual Signals for Efficient Video Large Language Models

• URL: http://arxiv.org/abs/2604.09547v2
• Date: Mon, 13 Apr 2026 06:42:59 GMT
• Title: Tango: Taming Visual Signals for Efficient Video Large Language Models
• Abstract summary: This work revisits and advances the two predominant token-pruning paradigms: attention-based selection and similarity-based clustering.<n>To address these bottlenecks, we propose Tango, a novel framework designed to optimize the utilization of visual signals.
• Score: 51.43125826972642
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Token pruning has emerged as a mainstream approach for developing efficient Video Large Language Models (Video LLMs). This work revisits and advances the two predominant token-pruning paradigms: attention-based selection and similarity-based clustering. Our study reveals two critical limitations in existing methods: (1) conventional top-k selection strategies fail to fully account for the attention distribution, which is often spatially multi-modal and long-tailed in magnitude; and (2) direct similarity-based clustering frequently generates fragmented clusters, resulting in distorted representations after pooling. To address these bottlenecks, we propose Tango, a novel framework designed to optimize the utilization of visual signals. Tango integrates a diversity-driven strategy to enhance attention-based token selection, and introduces Spatio-temporal Rotary Position Embedding (ST-RoPE) to preserve geometric structure via locality priors. Comprehensive experiments across various Video LLMs and video understanding benchmarks demonstrate the effectiveness and generalizability of our approach. Notably, when retaining only 10% of the video tokens, Tango preserves 98.9% of the original performance on LLaVA-OV while delivering a 1.88$\times$ inference speedup.

Related papers

• ST-SimDiff: Balancing Spatiotemporal Similarity and Difference for Efficient Video Understanding with MLLMs [20.712141528369553]
  We develop a training-free framework named ST-Sim-Diff to handle complex video relationships.<n>Our method significantly outperforms state-of-the-art approaches while substantially reducing computational costs.
  arXiv  Detail & Related papers  (2026-05-21T08:27:15Z)
• Beyond Pedestrians: Caption-Guided CLIP Framework for High-Difficulty Video-based Person Re-Identification [0.0]
  We propose a caption-guided CLIP framework that leverages textual descriptions and learnable tokens.<n>We evaluate our approach on two standard datasets and two newly constructed high-difficulty datasets.<n> Experimental results demonstrate that our method outperforms current state-of-art approaches.
  arXiv  Detail & Related papers  (2026-04-09T02:55:51Z)
• Cluster-Wise Spatio-Temporal Masking for Efficient Video-Language Pretraining [86.49790441700195]
  ClusterSTM is a Cluster-Wise Spatio-Temporal Masking strategy for efficient video-language pretraining.<n>We introduce a video-text relevance reconstruction objective that aligns high-level multimodal semantics beyond conventional visual reconstruction.
  arXiv  Detail & Related papers  (2026-03-24T08:48:15Z)
• Aligning Effective Tokens with Video Anomaly in Large Language Models [42.99603812716817]
  We propose VA-GPT, a novel MLLM designed for summarizing and localizing abnormal events in various videos.<n>Our approach efficiently aligns effective tokens between visual encoders and LLMs through two key proposed modules.<n>We construct an instruction-following dataset specifically for fine-tuning video-anomaly-aware MLLMs.
  arXiv  Detail & Related papers  (2025-08-08T14:30:05Z)
• Perceive Anything: Recognize, Explain, Caption, and Segment Anything in Images and Videos [53.723410664944566]
  We present Perceive Anything Model (PAM), a framework for comprehensive region-level visual understanding in images and videos.<n>Our approach extends the powerful segmentation model SAM 2 by integrating Large Language Models (LLMs), enabling simultaneous object segmentation.<n>A key component, Semantic Perceiver, is introduced to efficiently transform SAM 2's rich visual features into multi-modal tokens.
  arXiv  Detail & Related papers  (2025-06-05T17:51:39Z)
• ViaRL: Adaptive Temporal Grounding via Visual Iterated Amplification Reinforcement Learning [68.76048244253582]
  We introduce ViaRL, the first framework to leverage rule-based reinforcement learning (RL) for optimizing frame selection in video understanding.<n>ViaRL utilizes the answer accuracy of a downstream model as a reward signal to train a frame selector through trial-and-error.<n>ViaRL consistently delivers superior temporal grounding performance and robust generalization across diverse video understanding tasks.
  arXiv  Detail & Related papers  (2025-05-21T12:29:40Z)
• Multi-Cue Adaptive Visual Token Pruning for Large Vision-Language Models [85.51753014478315]
  We introduce AdaptPrune, a novel plug-and-play training-free pruning method.<n>It builds on conventional attention-based pruning by integrating spatial distance and token similarity with an adaptive NMS approach.<n>Our approach ensures a comprehensive evaluation of token importance and substantially refines the pruning decisions.
  arXiv  Detail & Related papers  (2025-03-11T03:58:17Z)
• Beyond Training: Dynamic Token Merging for Zero-Shot Video Understanding [11.211803499867639]
  We propose DYTO, a novel dynamic token merging framework for zero-shot video understanding.<n> DYTO integrates a hierarchical frame selection and a bipartite token merging strategy to dynamically cluster key frames and selectively compress token sequences.<n>Experiments demonstrate the effectiveness of DYTO, achieving superior performance compared to both fine-tuned and training-free methods.
  arXiv  Detail & Related papers  (2024-11-21T18:30:11Z)
• Group Contextualization for Video Recognition [80.3842253625557]
  Group contextualization (GC) can boost the performance of 2D-CNN (e.g., TSN) and TSM. GC embeds feature with four different kinds of contexts in parallel. Group contextualization can boost the performance of 2D-CNN (e.g., TSN) to a level comparable to the state-the-art video networks.
  arXiv  Detail & Related papers  (2022-03-18T01:49:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.