OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models

• URL: http://arxiv.org/abs/2607.23193v1
• Date: Sat, 25 Jul 2026 13:16:04 GMT
• Title: OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models
• Abstract summary: We propose a training-free token compression framework that uses the query as a shared semantic anchor while estimating relevance separately for audio and video.<n>Across four audio-video benchmarks and two Qwen2.5- Omni model scales, OmniScope achieves the best average accuracy across all compression settings.<n>These results suggest a simple design principle for OmniLLM inference: share the query across modalities, but not the salience estimates.
• Score: 4.202254591215664
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing token compression methods for omnimodal large language models typically rely on one modality to determine what to retain in the other. We show that this assumption often breaks down: for the same query, audio and video relevance often peaks at different moments. This cross-modal salience mismatch makes unidirectional guidance prone to discarding answer-critical cues under aggressive compression. We propose OmniScope, a training-free token compression framework that uses the query as a shared semantic anchor while estimating relevance separately for audio and video. OmniScope allocates modality-specific token budgets, prunes visual tokens with an anchor-delta strategy that preserves both global context and temporal changes, and merges audio tokens within each second to reduce redundancy while maintaining temporal continuity. Across four audio-video benchmarks and two Qwen2.5-Omni model scales, OmniScope achieves the best average accuracy across all compression settings. At 25% overall token retention, it delivers up to 3.53x prefill speedup and more than 15% GPU memory reduction, with only a 0.35-point drop in average accuracy. These results suggest a simple design principle for OmniLLM inference: share the query across modalities, but not the salience estimates. The code is available at https://github.com/MAC-AutoML/OmniScope.

Related papers

• Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models [65.49950267695267]
  We introduce Omni2LoRA, a framework for efficient parametric memory compression via coherence-preserving context distillation.<n>By converting multimodal memory into a fixed-budget, reusable parameter state, our method drives answer-time multimodal-token load to zero.
  arXiv  Detail & Related papers  (2026-08-10T07:49:10Z)
• VoxZip: Semantic-Anchored Temporal KV Cache Compression for Long-Context Audio Inference [53.883871460875234]
  VoxZip is a semantic-anchored KV cache compression framework.<n>First stage uses automatic speech recognition (ASR) transcriptions as explicit semantic anchors to temporally align, compress, and fuse audio tokens.<n>Second stage employs a dynamic filtering strategy based on temporally decayed accumulated attention to evict non-essential tokens.
  arXiv  Detail & Related papers  (2026-08-09T08:17:18Z)
• Omni-Prune: Query-Aware Unified Token Pruning for Efficient Omnimodal Large Language Models [18.902147033940434]
  Omni-Prune is a training-free, query-aware audio-visual token pruning framework.<n>It removes redundancy from both modalities while keeping task-relevant cross-modal evidence.<n>It delivers up to 3.25x prefill speedup and 1.3x memory reduction while retaining over 99% of full-model performance.
  arXiv  Detail & Related papers  (2026-07-26T03:51:39Z)
• OmniFocus: Query-Guided Modality-Balanced Token Compression for Omni-Modal Large Language Models [72.34361439042634]
  We propose a training-free query-guided token compression method for OmniLLMs that performs independent importance estimation for video and audio.<n>Experiments on the Qwen2.5- Omni model family across four audio-visual benchmarks show that OmniFocus maintains strong compressed performance at low token retention ratios.
  arXiv  Detail & Related papers  (2026-07-03T07:41:00Z)
• OmniSelect: Dynamic Modality-Aware Token Compression for Efficient Omni-modal Large Language Models [49.34557936004057]
  We propose a training-free, modality-adaptive token pruning framework that dynamically selects appropriate compression strategies for multimodal inputs.<n>Our method achieves efficient multimodal token reduction while maintaining strong performance, without requiring any additional training.
  arXiv  Detail & Related papers  (2026-05-18T08:33:52Z)
• OmniDrop: Layer-wise Token Pruning for Omni-modal LLMs via Query-Guidance [11.057999826097829]
  The token explosion caused by high-resolution audio and video inputs remains a critical bottleneck for real-time applications and long-form reasoning.<n>Existing omni-modal token compression methods typically prune tokens at the input embedding level, relying on audio-video similarity or temporal co-occurrence as proxies for semantic relevance.<n>We propose OmniDrop, a training-free, layer-wise token pruning framework that progressively prunes audiovisual tokens within the LLM decoder layers rather than at the input-level.
  arXiv  Detail & Related papers  (2026-05-14T06:54:37Z)
• OmniRefine: Alignment-Aware Cooperative Compression for Efficient Omnimodal Large Language Models [14.355322423471018]
  We propose a training-free two-stage framework for efficient audio-visual token compression in Omni-LLMs.<n>First, Correspondence-Preserving Chunk Refinement refines native chunk boundaries into cross-modally aligned compression units.<n>Second, Modality-Aware Cooperative Compression jointly compresses video and audio tokens within each refined unit to reduce redundancy while preserving critical evidence.
  arXiv  Detail & Related papers  (2026-05-12T12:42:44Z)
• Keep What Audio Cannot Say: Context-Preserving Token Pruning for Omni-LLMs [38.874953063372416]
  We propose ContextGuard as an inference-time token pruning framework.<n> ContextGuard predicts coarse visual semantics from audio and prunes video tokens.<n>It achieves full-token-level performance on five of six benchmarks while pruning 55% of input tokens.
  arXiv  Detail & Related papers  (2026-05-12T06:35:29Z)
• MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interaction [76.4461698685681]
  Mini-o 4.5 is our latest effort towards human-level realtime streaming interaction.<n>It is based on Omni-CPM, a unified streaming framework that aligns omni-modal perception and outputs along a shared temporal axis.<n>With a total of 9B parameters, Mini-o 4.5 approaches Gemini 2.5 Flash in vision computation capabilities, delivering stateofthe-art open computation performance scale.
  arXiv  Detail & Related papers  (2026-04-30T04:05:43Z)
• Small Vision-Language Models are Smart Compressors for Long Video Understanding [73.65465038390771]
  Long-form video understanding relies on intent-driven efficiency rather than greedily padded context.<n>We propose Tempo, an efficient query-aware framework compressing long videos for downstream understanding.<n>Tests show Tempo compresses hour-long videos substantially below theoretical limits, proving true long-form video understanding relies on intent-driven efficiency.
  arXiv  Detail & Related papers  (2026-04-09T11:40:25Z)
• VisionSelector: End-to-End Learnable Visual Token Compression for Efficient Multimodal LLMs [82.72388893596555]
  Multimodal Large Language Models (MLLMs) encounter significant computational and memory bottlenecks.<n>Previous token compression techniques are often constrained by rules that risk discarding critical information.<n>We reformulate token compression as a lightweight plug-and-play framework that reformulates token compression into an end-to-end learnable decision process.
  arXiv  Detail & Related papers  (2025-10-18T17:54:18Z)
• Prune Spatio-temporal Tokens by Semantic-aware Temporal Accumulation [89.88214896713846]
  STA score considers two critical factors: temporal redundancy and semantic importance. We apply the STA module to off-the-shelf video Transformers and Videowins. Results: Kinetics-400 and Something-Something V2 achieve 30% overshelf reduction with a negligible 0.2% accuracy drop.
  arXiv  Detail & Related papers  (2023-08-08T19:38:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.