MorphoQuant: Modality-Aware Quantization for Omni-modal Large Language Models

• URL: http://arxiv.org/abs/2606.04349v2
• Date: Fri, 05 Jun 2026 07:15:28 GMT
• Title: MorphoQuant: Modality-Aware Quantization for Omni-modal Large Language Models
• Abstract summary: We propose a modality-aware PTQ framework to preserve cross-modal morphology and mitigate outlier loss.<n>Distribution-Aware Bias Compensation (DABC) selectively absorbs long-tailed outliers into channel-wise biases.<n>We also propose Morphology-Directed Quantization Function Optimization (MDQFO) to co-optimize the quantization grid with the bias mask.
• Score: 57.87791117741788
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Conventional Post-Training Quantization (PTQ) methods struggle with 4-bit Omni-modal Large Language Models (OLLMs) due to the extreme distribution heterogeneity and disparate outlier patterns across modalities. To address this, we propose MorphoQuant, a modality-aware PTQ framework engineered to preserve cross-modal morphology and mitigate outlier loss. Specifically, we introduce Distribution-Aware Bias Compensation (DABC), which selectively absorbs long-tailed outliers into channel-wise biases. This mechanism safeguards outlier magnitudes while maintaining high-precision discretization for dense inliers, thereby preserving accurate discretization across diverse modal distribution. Complementing this, we propose Morphology-Directed Quantization Function Optimization (MDQFO) to co-optimize the quantization grid with the bias mask, ensuring fine-grained alignment across modalities. Extensive evaluations on Qwen2.5-Omni across benchmarks like MMMU and Video-MME demonstrate our approach's superiority. Notably, our W4A4 model achieves 76.63% on ScienceQA, significantly outperforming SOTA W4A4 methods and surprisingly surpassing the W4A16 baseline, which fully demonstrates the exceptional accuracy-efficiency trade-off of our framework.

Related papers

• Accuracy is Not Enough: A Divergence-Based Approach to Evaluate Fidelity Loss in Quantized LLMs [13.897073114465094]
  Deployment of Large Language Models (LLMs) on memory-constrained edge devices relies heavily on aggressive post-training quantization.<n>We introduce a distribution-sensitive evaluation framework quantifying information loss in quantized LLMs.<n>We quantify probability mass displacement and distributional drift relative to the BF16 reference, capturing predictive distribution changes not reflected in top-1 accuracy.
  arXiv  Detail & Related papers  (2026-09-07T15:49:52Z)
• Attribution-Guided and Coverage-Maximized Pruning for Structural MoE Compression [74.00650541246374]
  We propose a structural pruning framework tailored for MoE models.<n>Our approach reduces memory footprint by 5.27$times and consistently outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-06-16T06:53:27Z)
• Breaking Modality Heterogeneity in Low-Bit Quantization for Large Vision-Language Models [36.14224700486847]
  Low-bit post-training quantization (PTQ) is a pivotal technique for deploying Vision-Language Models (VLMs) on resource-constrained devices.<n>Existing PTQ methods often degrade VLMs' accuracy due to the heterogeneous activation distributions of text and vision modalities during quantization.<n>We propose SplitQ, a channel-Splitting-driven post-training Quantization framework.
  arXiv  Detail & Related papers  (2026-05-19T14:49:57Z)
• SOAR: Scale Optimization for Accurate Reconstruction in NVFP4 Quantization [19.022444007775896]
  NVFP4 has recently emerged as an efficient 4-bit microscaling format for large language models.<n>Existing methods often yield suboptimal performance due to inflexible scale selection and the coupled treatment of quantization and dequantization scales.<n>We propose Scale Optimization for Accurate Reconstruction (SOAR), a novel post-training quantization framework that improves the accuracy of NVFP4 quantization.
  arXiv  Detail & Related papers  (2026-05-12T15:13:18Z)
• Normalized Architectures are Natively 4-Bit [49.13186675123547]
  nGPT, an architecture that constrains weights and hidden representations to the unit hypersphere, is inherently more robust to low-precision arithmetic.<n>We validate this approach on both a 1.2B dense model and hybrid (Mamba-Transformer) MoE models of up to 3B/30B parameters.
  arXiv  Detail & Related papers  (2026-05-07T11:54:07Z)
• BATQuant: Outlier-resilient MXFP4 Quantization via Learnable Block-wise Optimization [34.40883652676667]
  Post-Training Quantization methods suffer from severe performance collapse when applied to MXFP4.<n>We propose BATQuant, which restricts transformations to align with MXFP to prevent cross-block outlier propagation.<n>We show that BATQuant establishes new state-of-the-art results under aggressive W4A4KV16 configurations.
  arXiv  Detail & Related papers  (2026-03-17T14:37:08Z)
• ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs [4.431548809730958]
  ARCQuant is a framework that boosts NVFP4 performance via Augmented Residual Channels.<n>We show that ARCQuant achieves state-of-the-art accuracy, comparable to full-precision baselines in perplexity and downstream tasks.
  arXiv  Detail & Related papers  (2026-01-12T12:27:22Z)
• AWPO: Enhancing Tool-Use of Large Language Models through Explicit Integration of Reasoning Rewards [60.2998874976509]
  We propose advantage-weighted policy optimization (AWPO) to integrate explicit reasoning rewards to enhance tool-use capability.<n>AWPO incorporates variance-aware gating and difficulty-aware weighting to adaptively modulate advantages from reasoning signals.<n>Experiments demonstrate that AWPO achieves state-of-the-art performance across standard tool-use benchmarks.
  arXiv  Detail & Related papers  (2025-12-22T08:07:00Z)
• Improving Block-Wise LLM Quantization by 4-bit Block-Wise Optimal Float (BOF4): Analysis and Variations [22.127873567034825]
  Large language models (LLMs) demand extensive memory capacity during both fine-tuning and inference.<n>Existing methods apply block-wise quantization techniques, such as NF4 and AF4, to the network weights.<n>We show that these quantization techniques incur suboptimal quantization errors.
  arXiv  Detail & Related papers  (2025-05-10T14:00:15Z)
• TCAQ-DM: Timestep-Channel Adaptive Quantization for Diffusion Models [49.65286242048452]
  We propose a novel method dubbed Timestep-Channel Adaptive Quantization for Diffusion Models (TCAQ-DM)<n>The proposed method substantially outperforms the state-of-the-art approaches in most cases.
  arXiv  Detail & Related papers  (2024-12-21T16:57:54Z)
• Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularization [62.15918574997175]
  It is known that language models contain outlier channels whose values on average are orders of magnitude higher than other channels. We propose a strategy which regularizes a layer's inputs via quantization-aware training (QAT) and its outputs via activation kurtosis regularization. We show that regularizing both the inputs and outputs is crucial for preventing a model's "migrating" the difficulty in input quantization to the weights.
  arXiv  Detail & Related papers  (2024-04-04T17:25:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.