QAPruner: Quantization-Aware Vision Token Pruning for Multimodal Large Language Models

• URL: http://arxiv.org/abs/2604.02816v1
• Date: Fri, 03 Apr 2026 07:32:07 GMT
• Title: QAPruner: Quantization-Aware Vision Token Pruning for Multimodal Large Language Models
• Abstract summary: Multimodal Large Language Models (MLLMs) have shown strong reasoning ability, but their high computational and memory costs hinder deployment in resource-constrained settings.<n>While Post-Training Quantization (PTQ) and vision token pruning are standard compression techniques, they are usually treated as independent optimizations.<n>In this paper, we show that these two techniques are strongly coupled: naively applying semantic-based token pruning to PTQ-optimized MLLMs can discard activation outliers that are important for numerical stability and thus worsen quantization errors in low-bit regimes.<n>We propose a quantization-aware vision token pruning framework
• Score: 18.353831760548267
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Multimodal Large Language Models (MLLMs) have shown strong reasoning ability, but their high computational and memory costs hinder deployment in resource-constrained settings. While Post-Training Quantization (PTQ) and vision token pruning are standard compression techniques, they are usually treated as independent optimizations. In this paper, we show that these two techniques are strongly coupled: naively applying semantic-based token pruning to PTQ-optimized MLLMs can discard activation outliers that are important for numerical stability and thus worsen quantization errors in low-bit regimes (\textit{e.g.}, W4A4). To address this issue, we propose a quantization-aware vision token pruning framework. Our method introduces a lightweight hybrid sensitivity metric that combines simulated group-wise quantization error with outlier intensity. By combining this metric with standard semantic relevance scores, the method retains tokens that are both semantically informative and robust to quantization. Experiments on standard LLaVA architectures show that our method consistently outperforms naive integration baselines. At an aggressive pruning ratio that retains only 12.5\% of visual tokens, our framework improves accuracy by 2.24\% over the baseline and even surpasses dense quantization without pruning. To the best of our knowledge, this is the first method that explicitly co-optimizes vision token pruning and PTQ for accurate low-bit MLLM inference.

Related papers

• TOPS: First-Principles Visual Token Pruning via Constructing Token Optimal Preservation Sets for Efficient MLLM Inference [54.41143483594129]
  Multimodal large language models (MLLMs) have achieved strong multimodal reasoning capabilities, but their efficiency is limited by the large number of visual tokens.<n>Visual token pruning offers a natural solution, yet existing methods are imperfect.<n>We propose TOPS, a training-free and model-agnostic pruning module that can be applied to various MLLMs.
  arXiv  Detail & Related papers  (2026-06-25T15:29:37Z)
• Towards Joint Quantization and Token Pruning of Vision-Language Models [53.978753457744055]
  Token pruning and low-bit quantization are complementary for reducing inference costs.<n>We present a collaborative quantization-and-pruning framework that unifies low-bit inference and deterministic visual-token pruning.<n> Experiments on standard VLM benchmarks show improved robustness over stage-wise baselines under the same low-bit regime.
  arXiv  Detail & Related papers  (2026-04-19T08:18:29Z)
• Rethinking Token Reduction for Large Vision-Language Models [95.48478689025696]
  Large-Language Models (LVLMs) excel in visual understanding and reasoning, but the excessive visual tokens lead to high inference costs.<n>We propose a learning-based prompt-agnostic, termed MetaCompress, overcoming the limitations of generalization designs.
  arXiv  Detail & Related papers  (2026-03-23T08:40:08Z)
• Fine-Grained Post-Training Quantization for Large Vision Language Models with Quantization-Aware Integrated Gradients [36.01708036519368]
  Large Vision Language Models (LVLMs) have achieved remarkable success in a range of downstream tasks that require multimodal interaction.<n>Existing LVLM quantization methods measure token sensitivity at the modality level, which fails to capture the complex cross-token interactions.<n>Inspired by axiomatic attribution in mechanistic interpretability, we introduce a fine-grained quantization strategy on Quantization-aware Integrated Gradients (QIG)<n>Our method improves the average accuracy of LLaVA-onevision-7B by 1.60%, reducing the gap to its full-precision counterpart to only 1.33%.
  arXiv  Detail & Related papers  (2026-03-18T15:03:43Z)
• IDPruner: Harmonizing Importance and Diversity in Visual Token Pruning for MLLMs [11.254129271889035]
  Visual token pruning has emerged as a critical technique for accelerating MLLM inference.<n>IDPruner achieves state-of-the-art performance and superior generalization across diverse architectures and tasks.
  arXiv  Detail & Related papers  (2026-02-10T11:20:24Z)
• VEQ: Modality-Adaptive Quantization for MoE Vision-Language Models [41.557274086591924]
  Post-Training Quantization (PTQ) is an effective training-free technique to address the massive memory and computation overhead.<n>Visual Expert Quantization (VEQ) is a dual-aware quantization framework designed to accommodate cross-modal differences and heterogeneity between experts.<n>Our method achieves significant average accuracy gains of 2.04% on Kimi-VL and 3.09% on Qwen3-VL compared to the previous SOTA quantization methods.
  arXiv  Detail & Related papers  (2026-02-01T05:53:09Z)
• FrugalPrompt: Reducing Contextual Overhead in Large Language Models via Token Attribution [3.4666771782038652]
  Large language models (LLMs) owe much of their stellar performance to expansive input contexts, yet such verbosity inflates monetary costs, carbon footprint, and inference-time latency.<n>We introduce FrugalPrompt, a novel prompt compression framework for LLMs, which retains only the most semantically significant tokens.<n>We evaluate the approach across four NLP tasks: Sentiment Analysis, Commonsense QA, Summarization, and Mathematical Reasoning.
  arXiv  Detail & Related papers  (2025-10-18T10:22:13Z)
• Winning the Pruning Gamble: A Unified Approach to Joint Sample and Token Pruning for Efficient Supervised Fine-Tuning [71.30276778807068]
  We propose a unified framework that strategically coordinates sample pruning and token pruning.<n>Q-Tuning achieves a +38% average improvement over the full-data SFT baseline using only 12.5% of the original training data.
  arXiv  Detail & Related papers  (2025-09-28T13:27:38Z)
• VLMQ: Efficient Post-Training Quantization for Large Vision-Language Models via Hessian Augmentation [8.891793681316992]
  Post-training quantization (PTQ) has emerged as an effective approach for compressing large models and accelerating their inference without retraining.<n>While PTQ has been extensively studied in the context of large language models (LLMs), its applicability to vision-language models (VLMs) remains underexplored.<n>We propose a novel importance-aware PTQ framework tailored for VLMs, dubbed VLMQ.
  arXiv  Detail & Related papers  (2025-08-05T11:57:03Z)
• MPQ-DMv2: Flexible Residual Mixed Precision Quantization for Low-Bit Diffusion Models with Temporal Distillation [74.34220141721231]
  We present MPQ-DMv2, an improved textbfMixed textbfPrecision textbfQuantization framework for extremely low-bit textbfDiffusion textbfModels.
  arXiv  Detail & Related papers  (2025-07-06T08:16:50Z)
• Pushing the Limits of Large Language Model Quantization via the Linearity Theorem [71.3332971315821]
  We present a "line theoremarity" establishing a direct relationship between the layer-wise $ell$ reconstruction error and the model perplexity increase due to quantization. This insight enables two novel applications: (1) a simple data-free LLM quantization method using Hadamard rotations and MSE-optimal grids, dubbed HIGGS, and (2) an optimal solution to the problem of finding non-uniform per-layer quantization levels.
  arXiv  Detail & Related papers  (2024-11-26T15:35:44Z)
• SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models [63.118592279833656]
  Post-training quantization (PTQ) is an effective technique for compressing large language models (LLMs)<n>We propose SliM-LLM, a salience-driven mixed-precision quantization framework that allocates bit-widths at the group-wise.<n> Experiments show that SliM-LLM achieves superior performance across various LLMs at low bit-widths.
  arXiv  Detail & Related papers  (2024-05-23T16:21:48Z)
• LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit [55.73370804397226]
  Quantization, a key compression technique, can effectively mitigate these demands by compressing and accelerating large language models. We present LLMC, a plug-and-play compression toolkit, to fairly and systematically explore the impact of quantization. Powered by this versatile toolkit, our benchmark covers three key aspects: calibration data, algorithms (three strategies), and data formats.
  arXiv  Detail & Related papers  (2024-05-09T11:49:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.