Where and How to Enhance: Discovering Bit-Width Contribution for Mixed Precision Quantization

• URL: http://arxiv.org/abs/2508.03002v1
• Date: Tue, 05 Aug 2025 02:14:21 GMT
• Title: Where and How to Enhance: Discovering Bit-Width Contribution for Mixed Precision Quantization
• Authors: Haidong Kang, Lianbo Ma, Guo Yu, Shangce Gao,
• Abstract summary: Mixed precision quantization (MPQ) is an effective quantization approach to achieve accuracy-complexity trade-off of neural network.<n>We propose a Shapley-based MPQ (SMPQ) method, which measures the bit-width operation direct contribution on the MPQ task.
• Score: 10.315643425890286
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Mixed precision quantization (MPQ) is an effective quantization approach to achieve accuracy-complexity trade-off of neural network, through assigning different bit-widths to network activations and weights in each layer. The typical way of existing MPQ methods is to optimize quantization policies (i.e., bit-width allocation) in a gradient descent manner, termed as Differentiable (DMPQ). At the end of the search, the bit-width associated to the quantization parameters which has the largest value will be selected to form the final mixed precision quantization policy, with the implicit assumption that the values of quantization parameters reflect the operation contribution to the accuracy improvement. While much has been discussed about the MPQ improvement, the bit-width selection process has received little attention. We study this problem and argue that the magnitude of quantization parameters does not necessarily reflect the actual contribution of the bit-width to the task performance. Then, we propose a Shapley-based MPQ (SMPQ) method, which measures the bit-width operation direct contribution on the MPQ task. To reduce computation cost, a Monte Carlo sampling-based approximation strategy is proposed for Shapley computation. Extensive experiments on mainstream benchmarks demonstrate that our SMPQ consistently achieves state-of-the-art performance than gradient-based competitors.

Related papers

• FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximation [55.12070409045766]
  Post-training quantization (PTQ) has stood out as a cost-effective and promising model compression paradigm in recent years.<n>Current PTQ methods for Vision Transformers (ViTs) still suffer from significant accuracy degradation, especially under low-bit quantization.
  arXiv  Detail & Related papers  (2025-06-13T07:57:38Z)
• Deep Unfolding with Kernel-based Quantization in MIMO Detection [26.033613526407226]
  This paper proposes a novel kernel-based adaptive quantization (KAQ) framework for deep unfolding networks.<n>The accuracy of proposed KAQ framework outperforms traditional methods and successfully reduces the model's inference latency.
  arXiv  Detail & Related papers  (2025-05-19T05:50:24Z)
• Mixed-Precision Graph Neural Quantization for Low Bit Large Language Models [13.709080134204326]
  Post-Training Quantization (PTQ) is pivotal for deploying large language models within resource-limited settings.<n>We introduce a Mixed-precision Graph Neural PTQ (MG-PTQ) approach, employing a graph neural network (GNN) module to capture dependencies among weights.<n>Our method more effectively captures dependencies among target weights, leading to a more accurate assessment of weight importance.
  arXiv  Detail & Related papers  (2025-01-30T05:39:01Z)
• Channel-Wise Mixed-Precision Quantization for Large Language Models [47.00361921910259]
  Large Language Models (LLMs) have demonstrated remarkable success across a wide range of language tasks.<n>Weight-only quantization presents a promising solution to reduce the memory footprint of LLMs.<n>We introduce Channel-Wise Mixed-Precision Quantization (CMPQ), a novel mixed-precision quantization method.
  arXiv  Detail & Related papers  (2024-10-16T21:34:41Z)
• SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models [63.118592279833656]
  Post-training quantization (PTQ) is an effective technique for compressing large language models (LLMs)<n>We propose SliM-LLM, a salience-driven mixed-precision quantization framework that allocates bit-widths at the group-wise.<n> Experiments show that SliM-LLM achieves superior performance across various LLMs at low bit-widths.
  arXiv  Detail & Related papers  (2024-05-23T16:21:48Z)
• On-Chip Hardware-Aware Quantization for Mixed Precision Neural Networks [52.97107229149988]
  We propose an On-Chip Hardware-Aware Quantization framework, performing hardware-aware mixed-precision quantization on deployed edge devices. For efficiency metrics, we built an On-Chip Quantization Aware pipeline, which allows the quantization process to perceive the actual hardware efficiency of the quantization operator. For accuracy metrics, we propose Mask-Guided Quantization Estimation technology to effectively estimate the accuracy impact of operators in the on-chip scenario.
  arXiv  Detail & Related papers  (2023-09-05T04:39:34Z)
• Mixed-Precision Quantization for Deep Vision Models with Integer Quadratic Programming [7.0146264551420066]
  Quantization is a widely used technique to compress neural networks.<n>MPQ addresses this by assigning varied bit-widths to layers, optimizing the accuracy-efficiency trade-off.<n>We introduce CLADO, a practical sensitivity-based MPQ algorithm that captures crosslayer dependency of quantization error.
  arXiv  Detail & Related papers  (2023-07-11T15:56:00Z)
• SDQ: Stochastic Differentiable Quantization with Mixed Precision [46.232003346732064]
  We present a novel Differentiable Quantization (SDQ) method that can automatically learn the MPQ strategy. After the optimal MPQ strategy is acquired, we train our network with entropy-aware bin regularization and knowledge distillation. SDQ outperforms all state-of-the-art mixed datasets or single precision quantization with a lower bitwidth.
  arXiv  Detail & Related papers  (2022-06-09T12:38:18Z)
• Cluster-Promoting Quantization with Bit-Drop for Minimizing Network Quantization Loss [61.26793005355441]
  Cluster-Promoting Quantization (CPQ) finds the optimal quantization grids for neural networks. DropBits is a new bit-drop technique that revises the standard dropout regularization to randomly drop bits instead of neurons. We experimentally validate our method on various benchmark datasets and network architectures.
  arXiv  Detail & Related papers  (2021-09-05T15:15:07Z)
• BSQ: Exploring Bit-Level Sparsity for Mixed-Precision Neural Network Quantization [32.770842274996774]
  Mixed-precision quantization can potentially achieve the optimal tradeoff between performance and compression rate of deep neural networks. Previous methods either examine only a small manually-designed search space or utilize a cumbersome neural architecture search to explore the vast search space. This work proposes bit-level sparsity quantization (BSQ) to tackle the mixed-precision quantization from a new angle of inducing bit-level sparsity.
  arXiv  Detail & Related papers  (2021-02-20T22:37:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.