Data-Free Quantization via Mixed-Precision Compensation without Fine-Tuning

• URL: http://arxiv.org/abs/2307.00498v1
• Date: Sun, 2 Jul 2023 07:16:29 GMT
• Title: Data-Free Quantization via Mixed-Precision Compensation without Fine-Tuning
• Authors: Jun Chen, Shipeng Bai, Tianxin Huang, Mengmeng Wang, Guanzhong Tian, Yong Liu
• Abstract summary: We propose a data-free mixed-precision compensation (DF-MPC) method to recover the performance of an ultra-low precision quantized model without any data and fine-tuning process. Our DF-MPC is able to achieve higher accuracy for an ultra-low precision quantized model compared to the recent methods without any data and fine-tuning process.
• Score: 20.413801240717646
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural network quantization is a very promising solution in the field of model compression, but its resulting accuracy highly depends on a training/fine-tuning process and requires the original data. This not only brings heavy computation and time costs but also is not conducive to privacy and sensitive information protection. Therefore, a few recent works are starting to focus on data-free quantization. However, data-free quantization does not perform well while dealing with ultra-low precision quantization. Although researchers utilize generative methods of synthetic data to address this problem partially, data synthesis needs to take a lot of computation and time. In this paper, we propose a data-free mixed-precision compensation (DF-MPC) method to recover the performance of an ultra-low precision quantized model without any data and fine-tuning process. By assuming the quantized error caused by a low-precision quantized layer can be restored via the reconstruction of a high-precision quantized layer, we mathematically formulate the reconstruction loss between the pre-trained full-precision model and its layer-wise mixed-precision quantized model. Based on our formulation, we theoretically deduce the closed-form solution by minimizing the reconstruction loss of the feature maps. Since DF-MPC does not require any original/synthetic data, it is a more efficient method to approximate the full-precision model. Experimentally, our DF-MPC is able to achieve higher accuracy for an ultra-low precision quantized model compared to the recent methods without any data and fine-tuning process.

Related papers

• Guaranteed Approximation Bounds for Mixed-Precision Neural Operators [83.64404557466528]
  We build on intuition that neural operator learning inherently induces an approximation error. We show that our approach reduces GPU memory usage by up to 50% and improves throughput by 58% with little or no reduction in accuracy.
  arXiv  Detail & Related papers  (2023-07-27T17:42:06Z)
• Post-training Model Quantization Using GANs for Synthetic Data Generation [57.40733249681334]
  We investigate the use of synthetic data as a substitute for the calibration with real data for the quantization method. We compare the performance of models quantized using data generated by StyleGAN2-ADA and our pre-trained DiStyleGAN, with quantization using real data and an alternative data generation method based on fractal images.
  arXiv  Detail & Related papers  (2023-05-10T11:10:09Z)
• Mixed-Precision Inference Quantization: Radically Towards Faster inference speed, Lower Storage requirement, and Lower Loss [4.877532217193618]
  Existing quantization techniques rely heavily on experience and "fine-tuning" skills. This study provides a methodology for acquiring a mixed-precise quantization model with a lower loss than the full precision model. In particular, we will demonstrate that neural networks with massive identity mappings are resistant to the quantization method.
  arXiv  Detail & Related papers  (2022-07-20T10:55:34Z)
• ClusterQ: Semantic Feature Distribution Alignment for Data-Free Quantization [111.12063632743013]
  We propose a new and effective data-free quantization method termed ClusterQ. To obtain high inter-class separability of semantic features, we cluster and align the feature distribution statistics. We also incorporate the intra-class variance to solve class-wise mode collapse.
  arXiv  Detail & Related papers  (2022-04-30T06:58:56Z)
• Mixed Precision Low-bit Quantization of Neural Network Language Models for Speech Recognition [67.95996816744251]
  State-of-the-art language models (LMs) represented by long-short term memory recurrent neural networks (LSTM-RNNs) and Transformers are becoming increasingly complex and expensive for practical applications. Current quantization methods are based on uniform precision and fail to account for the varying performance sensitivity at different parts of LMs to quantization errors. Novel mixed precision neural network LM quantization methods are proposed in this paper.
  arXiv  Detail & Related papers  (2021-11-29T12:24:02Z)
• Diverse Sample Generation: Pushing the Limit of Data-free Quantization [85.95032037447454]
  This paper presents a generic Diverse Sample Generation scheme for the generative data-free post-training quantization and quantization-aware training. For large-scale image classification tasks, our DSG can consistently outperform existing data-free quantization methods.
  arXiv  Detail & Related papers  (2021-09-01T07:06:44Z)
• Zero-shot Adversarial Quantization [11.722728148523366]
  We propose a zero-shot adversarial quantization (ZAQ) framework, facilitating effective discrepancy estimation and knowledge transfer. This is achieved by a novel two-level discrepancy modeling to drive a generator to synthesize informative and diverse data examples. We conduct extensive experiments on three fundamental vision tasks, demonstrating the superiority of ZAQ over the strong zero-shot baselines.
  arXiv  Detail & Related papers  (2021-03-29T01:33:34Z)
• DAQ: Distribution-Aware Quantization for Deep Image Super-Resolution Networks [49.191062785007006]
  Quantizing deep convolutional neural networks for image super-resolution substantially reduces their computational costs. Existing works either suffer from a severe performance drop in ultra-low precision of 4 or lower bit-widths, or require a heavy fine-tuning process to recover the performance. We propose a novel distribution-aware quantization scheme (DAQ) which facilitates accurate training-free quantization in ultra-low precision.
  arXiv  Detail & Related papers  (2020-12-21T10:19:42Z)
• VecQ: Minimal Loss DNN Model Compression With Vectorized Weight Quantization [19.66522714831141]
  We develop a new quantization solution called VecQ, which can guarantee minimal direct quantization loss and better model accuracy. In addition, in order to up the proposed quantization process during training, we accelerate the quantization process with a parameterized estimation and probability-based calculation.
  arXiv  Detail & Related papers  (2020-05-18T07:38:44Z)
• Generative Low-bitwidth Data Free Quantization [44.613912463011545]
  We propose Generative Low-bitwidth Data Free Quantization (GDFQ) to remove the data dependence burden. With the help of generated data, we can quantize a model by learning knowledge from the pre-trained model. Our method achieves much higher accuracy on 4-bit quantization than the existing data free quantization method.
  arXiv  Detail & Related papers  (2020-03-07T16:38:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.