EntropyPrune: Matrix Entropy Guided Visual Token Pruning for Multimodal Large Language Models

• URL: http://arxiv.org/abs/2602.17196v1
• Date: Thu, 19 Feb 2026 09:29:43 GMT
• Title: EntropyPrune: Matrix Entropy Guided Visual Token Pruning for Multimodal Large Language Models
• Abstract summary: We introduce a matrix-entropy perspective and identify an "Entropy Collapse Layer" (ECL), where the information content of visual representations exhibits a sharp and consistent drop.<n>We propose EntropyPrune, a novel matrix-entropy-guided token pruning framework that quantifies the information value of individual tokens and prunes redundant ones.<n>Experiments on diverse multimodal benchmarks demonstrate that EntropyPrune consistently outperforms state-of-the-art pruning methods in both accuracy and efficiency.
• Score: 35.12029704777804
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal large language models (MLLMs) incur substantial inference cost due to the processing of hundreds of visual tokens per image. Although token pruning has proven effective for accelerating inference, determining when and where to prune remains largely heuristic. Existing approaches typically rely on static, empirically selected layers, which limit interpretability and transferability across models. In this work, we introduce a matrix-entropy perspective and identify an "Entropy Collapse Layer" (ECL), where the information content of visual representations exhibits a sharp and consistent drop, which provides a principled criterion for selecting the pruning stage. Building on this observation, we propose EntropyPrune, a novel matrix-entropy-guided token pruning framework that quantifies the information value of individual visual tokens and prunes redundant ones without relying on attention maps. Moreover, to enable efficient computation, we exploit the spectral equivalence of dual Gram matrices, reducing the complexity of entropy computation and yielding up to a 64x theoretical speedup. Extensive experiments on diverse multimodal benchmarks demonstrate that EntropyPrune consistently outperforms state-of-the-art pruning methods in both accuracy and efficiency. On LLaVA-1.5-7B, our method achieves a 68.2% reduction in FLOPs while preserving 96.0% of the original performance. Furthermore, EntropyPrune generalizes effectively to high-resolution and video-based models, highlighting the strong robustness and scalability in practical MLLM acceleration. The code will be publicly available at https://github.com/YahongWang1/EntropyPrune.

Related papers

• Post-Training Pruning for Diffusion Transformers [23.732968732627825]
  Diffusion Transformers (DiTs) have demonstrated impressive performance in image generation but suffer from substantial computational overhead and resource consumption.<n>Traditional pruning methods amplify the relative contribution of weights in the saliency metric.<n>We propose DiT-Pruning, which improves pruning performance by introducing customized saliency criteria and pruning granularity.
  arXiv  Detail & Related papers  (2026-07-01T13:30:49Z)
• ERA: Entropy-Guided Visual Token Pruning with Rectified Attention for Efficient MLLMs [77.57757084153083]
  We propose ERA, an Entropy-guided visual token pruning framework with Rectified Attention for efficient MLLMs.<n> ERA comprises three crucial components: Dual-view Entropy Pruning (DEP), Bias-aware Token Recycling (BTR), and Logit-preserving Attention Rectification (LAR)
  arXiv  Detail & Related papers  (2026-06-30T17:20:29Z)
• IWP: Token Pruning as Implicit Weight Pruning in Large Vision Language Models [14.898443090114142]
  Large Vision Language Models show impressive performance across image and video understanding tasks, yet their computational cost grows rapidly with the number of visual tokens.<n>Existing token pruning methods mitigate this issue through empirical approaches while overlooking the internal mechanism of attention.<n>We propose a novel training free token pruning framework grounded in the dual form perspective of attention.
  arXiv  Detail & Related papers  (2026-04-01T11:23:16Z)
• OTPrune: Distribution-Aligned Visual Token Pruning via Optimal Transport [20.586206895801258]
  Multi-modal large language models (MLLMs) achieve strong visual-language reasoning but suffer from high inference cost due to redundant visual tokens.<n>Recent work explores visual token pruning to accelerate inference, while existing pruning methods overlook the underlying distributional structure of visual representations.<n>We propose OTPrune, a training-free framework that formulates pruning as distribution alignment via optimal transport.
  arXiv  Detail & Related papers  (2026-02-22T21:02:47Z)
• IDPruner: Harmonizing Importance and Diversity in Visual Token Pruning for MLLMs [11.254129271889035]
  Visual token pruning has emerged as a critical technique for accelerating MLLM inference.<n>IDPruner achieves state-of-the-art performance and superior generalization across diverse architectures and tasks.
  arXiv  Detail & Related papers  (2026-02-10T11:20:24Z)
• Beyond Structure: Invariant Crystal Property Prediction with Pseudo-Particle Ray Diffraction [23.89478649565297]
  Crystal property prediction is computationally prohibitive to solve exactly for large many-body systems using traditional density functional theory.<n>We introduce PRDNet that leverages unique reciprocal-space diffraction besides graph representations.<n>Extensive experiments are conducted on Materials Project, JARVIS-DFT, and MatBench, demonstrating that the proposed model achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-09-26T02:30:23Z)
• PIP: Perturbation-based Iterative Pruning for Large Language Models [15.00536465178398]
  We propose PIP (Perturbation-based Iterative Pruning), a novel double-view structured pruning method to optimize Large Language Models.<n>With the calculation of gradient differences, PIP iteratively prunes those that struggle to distinguish between these two views.<n>Our experiments show that PIP reduces the parameter count by approximately 20% while retaining over 85% of the original model's accuracy.
  arXiv  Detail & Related papers  (2025-01-25T17:10:50Z)
• Q-VLM: Post-training Quantization for Large Vision-Language Models [73.19871905102545]
  We propose a post-training quantization framework of large vision-language models (LVLMs) for efficient multi-modal inference.<n>We mine the cross-layer dependency that significantly influences discretization errors of the entire vision-language model, and embed this dependency into optimal quantization strategy.<n> Experimental results demonstrate that our method compresses the memory by 2.78x and increase generate speed by 1.44x about 13B LLaVA model without performance degradation.
  arXiv  Detail & Related papers  (2024-10-10T17:02:48Z)
• Bypass Back-propagation: Optimization-based Structural Pruning for Large Language Models via Policy Gradient [57.9629676017527]
  We propose an optimization-based structural pruning that learns the pruning masks in a probabilistic space directly by optimizing the loss of the pruned model.<n>We achieve this by learning an underlying Bernoulli distribution to sample binary pruning masks.<n>Experiments conducted on LLaMA, LLaMA-2, LLaMA-3, Vicuna, and Mistral models demonstrate the promising performance of our method in efficiency and effectiveness.
  arXiv  Detail & Related papers  (2024-06-15T09:31:03Z)
• Data-free Weight Compress and Denoise for Large Language Models [96.68582094536032]
  We propose a novel approach termed Data-free Joint Rank-k Approximation for compressing the parameter matrices.<n>We achieve a model pruning of 80% parameters while retaining 93.43% of the original performance without any calibration data.
  arXiv  Detail & Related papers  (2024-02-26T05:51:47Z)
• ECoFLaP: Efficient Coarse-to-Fine Layer-Wise Pruning for Vision-Language Models [70.45441031021291]
  Large Vision-Language Models (LVLMs) can understand the world comprehensively by integrating rich information from different modalities. LVLMs are often problematic due to their massive computational/energy costs and carbon consumption. We propose Efficient Coarse-to-Fine LayerWise Pruning (ECoFLaP), a two-stage coarse-to-fine weight pruning approach for LVLMs.
  arXiv  Detail & Related papers  (2023-10-04T17:34:00Z)
• A Simplified Framework for Contrastive Learning for Node Representations [2.277447144331876]
  We investigate the potential of deploying contrastive learning in combination with Graph Neural Networks for embedding nodes in a graph. We show that the quality of the resulting embeddings and training time can be significantly improved by a simple column-wise postprocessing of the embedding matrix. This modification yields improvements in downstream classification tasks of up to 1.5% and even beats existing state-of-the-art approaches on 6 out of 8 different benchmarks.
  arXiv  Detail & Related papers  (2023-05-01T02:04:36Z)
• Efficient Iterative Amortized Inference for Learning Symmetric and Disentangled Multi-Object Representations [8.163697683448811]
  We introduce EfficientMORL, an efficient framework for the unsupervised learning of object-centric representations. We show that optimization challenges caused by requiring both symmetry and disentanglement can be addressed by high-cost iterative amortized inference. We demonstrate strong object decomposition and disentanglement on the standard multi-object benchmark while achieving nearly an order of magnitude faster training and test time inference.
  arXiv  Detail & Related papers  (2021-06-07T14:02:49Z)
• Efficient Semi-Implicit Variational Inference [65.07058307271329]
  We propose an efficient and scalable semi-implicit extrapolational (SIVI) Our method maps SIVI's evidence to a rigorous inference of lower gradient values.
  arXiv  Detail & Related papers  (2021-01-15T11:39:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.