IPCV: Information-Preserving Compression for MLLM Visual Encoders

• URL: http://arxiv.org/abs/2512.18747v1
• Date: Sun, 21 Dec 2025 14:28:28 GMT
• Title: IPCV: Information-Preserving Compression for MLLM Visual Encoders
• Authors: Yuan Chen, Zichen Wen, Yuzhou Wu, Xuyang Liu, Shuang Chen, Junpeng Ma, Weijia Li, Conghui He, Linfeng Zhang,
• Abstract summary: IPCV is a training-free, information-preserving compression framework for MLLM visual encoders.<n>We introduce Attention Stabilization (AS) to further alleviate the negative influence from token pruning.<n>IPCV substantially reduces end-to-end computation and outperforms state-of-the-art training-free token compression methods.
• Score: 44.76073540999133
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal Large Language Models (MLLMs) deliver strong vision-language performance but at high computational cost, driven by numerous visual tokens processed by the Vision Transformer (ViT) encoder. Existing token pruning strategies are inadequate: LLM-stage token pruning overlooks the ViT's overhead, while conventional ViT token pruning, without language guidance, risks discarding textually critical visual cues and introduces feature distortions amplified by the ViT's bidirectional attention. To meet these challenges, we propose IPCV, a training-free, information-preserving compression framework for MLLM visual encoders. IPCV enables aggressive token pruning inside the ViT via Neighbor-Guided Reconstruction (NGR) that temporarily reconstructs pruned tokens to participate in attention with minimal overhead, then fully restores them before passing to the LLM. Besides, we introduce Attention Stabilization (AS) to further alleviate the negative influence from token pruning by approximating the K/V of pruned tokens. It can be directly applied to previous LLM-side token pruning methods to enhance their performance. Extensive experiments show that IPCV substantially reduces end-to-end computation and outperforms state-of-the-art training-free token compression methods across diverse image and video benchmarks. Our code is available at https://github.com/Perkzi/IPCV.

Related papers

• EvoPrune: Early-Stage Visual Token Pruning for Efficient MLLMs [11.74199552426475]
  EvoPrune is an early-stage visual token pruning method for MLLMs that performs pruning directly during visual encoding.<n>EvoPrune achieves 2$times$ inference speedup with less than 1% performance degradation, demonstrating its potential for latency-sensitive MLLM deployment.
  arXiv  Detail & Related papers  (2026-03-04T03:15:14Z)
• ViTCoP: Accelerating Large Vision-Language Models via Visual and Textual Semantic Collaborative Pruning [8.933549837045932]
  Large Vision-Language Models incur high computational costs due to significant redundancy in their visual tokens.<n>We propose a Visual and Textual Semantic Collaborative Pruning framework (ViTCoP) that combines redundancy filtering in the vision encoder with step-wise co-pruning within the Large Language Models.
  arXiv  Detail & Related papers  (2026-01-25T12:47:30Z)
• Towards Lossless Ultimate Vision Token Compression for VLMs [11.485425012979052]
  Lossless Ultimate Vision tokens Compression (LUVC) framework is proposed.<n>LUVC compresses visual tokens until complete elimination at the final layer of language model.<n>Experiments show that LUVC achieves a 2 speedup inference in language model with negligible accuracy degradation.
  arXiv  Detail & Related papers  (2025-12-09T15:40:13Z)
• CoViPAL: Layer-wise Contextualized Visual Token Pruning for Large Vision-Language Models [75.88232735646018]
  Large Vision-Language Models (LVLMs) process multimodal inputs consisting of text tokens and vision tokens extracted from images or videos.<n>Existing methods attempt to prune redundant vision tokens, revealing substantial redundancy in visual representations.<n>We propose CoViPAL, a layer-wise contextualized visual token pruning method that employs a Plug-and-Play Pruning Module (PPM) to predict and remove redundant vision tokens before they are processed by the LVLM.
  arXiv  Detail & Related papers  (2025-08-24T07:47:00Z)
• VScan: Rethinking Visual Token Reduction for Efficient Large Vision-Language Models [57.2662376527586]
  VScan is a two-stage visual token reduction framework.<n>It addresses token redundancy by: (1) integrating complementary global and local scans with token merging during visual encoding, and (2) introducing pruning at intermediate layers of the language model.<n>VScan achieves a 2.91$times$ speedup in prefilling and a 10$times$ reduction in FLOPs, while retaining 95.4% of the original performance.
  arXiv  Detail & Related papers  (2025-05-28T17:59:08Z)
• [CLS] Token Tells Everything Needed for Training-free Efficient MLLMs [66.5266435598799]
  Multi-language Large Language Models (MLLMs) have recently demonstrated strong performance across a wide range of vision tasks.<n>However, their efficient deployment remains a substantial challenge due to high computational costs and memory requirements.<n>We introduce a simple yet effective method for train-free visual compression, called VTC- compression.
  arXiv  Detail & Related papers  (2024-12-08T05:29:39Z)
• Efficient Multi-modal Large Language Models via Visual Token Grouping [55.482198808206284]
  High-resolution images and videos pose a barrier to their broader adoption.<n> compressing vision tokens in MLLMs has emerged as a promising approach to reduce inference costs.<n>We introduce VisToG, a novel grouping mechanism that leverages the capabilities of pre-trained vision encoders to group similar image segments.
  arXiv  Detail & Related papers  (2024-11-26T09:36:02Z)
• Inference Optimal VLMs Need Fewer Visual Tokens and More Parameters [54.01228554126122]
  Vision Language Models (VLMs) have demonstrated strong capabilities across various visual understanding and reasoning tasks.<n>To reduce inference costs, one can either downsize the Large Language Models (LLMs) or reduce the number of input tokens needed to represent the image.<n>We take the first steps toward designing token compression algorithms tailored for high-compression settings.
  arXiv  Detail & Related papers  (2024-11-05T18:54:21Z)
• VoCo-LLaMA: Towards Vision Compression with Large Language Models [31.398537194299752]
  Vision-Language Models (VLMs) have achieved remarkable success in various multi-modal tasks, but they are often bottlenecked by the limited context window.<n>We propose VoCo-LLaMA, the first approach to compress vision tokens using LLMs.<n>Our method achieves minimal performance loss with a compression ratio of 576$times$, resulting in up to 94.8$%$ fewer FLOPs and 69.6$%$ acceleration in inference time.
  arXiv  Detail & Related papers  (2024-06-18T05:05:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.