A Generalized Zero-Shot Quantization of Deep Convolutional Neural
Networks via Learned Weights Statistics
- URL: http://arxiv.org/abs/2112.02834v1
- Date: Mon, 6 Dec 2021 07:41:16 GMT
- Title: A Generalized Zero-Shot Quantization of Deep Convolutional Neural
Networks via Learned Weights Statistics
- Authors: Prasen Kumar Sharma, Arun Abraham, Vikram Nelvoy Rajendiran
- Abstract summary: Quantizing floating-point weights and activations of deep convolutional neural networks to fixed-point representation yields reduced memory footprints and inference time.
Recently, efforts have been afoot towards zero-shot quantization that does not require original unlabelled training samples of a given task.
We propose a generalized zero-shot quantization (GZSQ) framework that neither requires original data nor relies on BN layer statistics.
- Score: 1.2891210250935146
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Quantizing the floating-point weights and activations of deep convolutional
neural networks to fixed-point representation yields reduced memory footprints
and inference time. Recently, efforts have been afoot towards zero-shot
quantization that does not require original unlabelled training samples of a
given task. These best-published works heavily rely on the learned batch
normalization (BN) parameters to infer the range of the activations for
quantization. In particular, these methods are built upon either empirical
estimation framework or the data distillation approach, for computing the range
of the activations. However, the performance of such schemes severely degrades
when presented with a network that does not accommodate BN layers. In this line
of thought, we propose a generalized zero-shot quantization (GZSQ) framework
that neither requires original data nor relies on BN layer statistics. We have
utilized the data distillation approach and leveraged only the pre-trained
weights of the model to estimate enriched data for range calibration of the
activations. To the best of our knowledge, this is the first work that utilizes
the distribution of the pretrained weights to assist the process of zero-shot
quantization. The proposed scheme has significantly outperformed the existing
zero-shot works, e.g., an improvement of ~ 33% in classification accuracy for
MobileNetV2 and several other models that are w & w/o BN layers, for a variety
of tasks. We have also demonstrated the efficacy of the proposed work across
multiple open-source quantization frameworks. Importantly, our work is the
first attempt towards the post-training zero-shot quantization of futuristic
unnormalized deep neural networks.
Related papers
- Post-Training Quantization for Re-parameterization via Coarse & Fine
Weight Splitting [13.270381125055275]
We propose a coarse & fine weight splitting (CFWS) method to reduce quantization error of weight.
We develop an improved KL metric to determine optimal quantization scales for activation.
For example, the quantized RepVGG-A1 model exhibits a mere 0.3% accuracy loss.
arXiv Detail & Related papers (2023-12-17T02:31:20Z) - Vertical Layering of Quantized Neural Networks for Heterogeneous
Inference [57.42762335081385]
We study a new vertical-layered representation of neural network weights for encapsulating all quantized models into a single one.
We can theoretically achieve any precision network for on-demand service while only needing to train and maintain one model.
arXiv Detail & Related papers (2022-12-10T15:57:38Z) - BiTAT: Neural Network Binarization with Task-dependent Aggregated
Transformation [116.26521375592759]
Quantization aims to transform high-precision weights and activations of a given neural network into low-precision weights/activations for reduced memory usage and computation.
Extreme quantization (1-bit weight/1-bit activations) of compactly-designed backbone architectures results in severe performance degeneration.
This paper proposes a novel Quantization-Aware Training (QAT) method that can effectively alleviate performance degeneration.
arXiv Detail & Related papers (2022-07-04T13:25:49Z) - Cluster-Promoting Quantization with Bit-Drop for Minimizing Network
Quantization Loss [61.26793005355441]
Cluster-Promoting Quantization (CPQ) finds the optimal quantization grids for neural networks.
DropBits is a new bit-drop technique that revises the standard dropout regularization to randomly drop bits instead of neurons.
We experimentally validate our method on various benchmark datasets and network architectures.
arXiv Detail & Related papers (2021-09-05T15:15:07Z) - Zero-shot Adversarial Quantization [11.722728148523366]
We propose a zero-shot adversarial quantization (ZAQ) framework, facilitating effective discrepancy estimation and knowledge transfer.
This is achieved by a novel two-level discrepancy modeling to drive a generator to synthesize informative and diverse data examples.
We conduct extensive experiments on three fundamental vision tasks, demonstrating the superiority of ZAQ over the strong zero-shot baselines.
arXiv Detail & Related papers (2021-03-29T01:33:34Z) - Direct Quantization for Training Highly Accurate Low Bit-width Deep
Neural Networks [73.29587731448345]
This paper proposes two novel techniques to train deep convolutional neural networks with low bit-width weights and activations.
First, to obtain low bit-width weights, most existing methods obtain the quantized weights by performing quantization on the full-precision network weights.
Second, to obtain low bit-width activations, existing works consider all channels equally.
arXiv Detail & Related papers (2020-12-26T15:21:18Z) - Recurrence of Optimum for Training Weight and Activation Quantized
Networks [4.103701929881022]
Training deep learning models with low-precision weights and activations involves a demanding optimization task.
We show how to overcome the nature of network quantization.
We also show numerical evidence of the recurrence phenomenon of weight evolution in training quantized deep networks.
arXiv Detail & Related papers (2020-12-10T09:14:43Z) - Optimal Quantization for Batch Normalization in Neural Network
Deployments and Beyond [18.14282813812512]
Batch Normalization (BN) poses a challenge for Quantized Neural Networks (QNNs)
We propose a novel method to quantize BN by converting an affine transformation of two floating points to a fixed-point operation with shared quantized scale.
Our method is verified by experiments at layer level on CIFAR and ImageNet datasets.
arXiv Detail & Related papers (2020-08-30T09:33:29Z) - Revisiting Initialization of Neural Networks [72.24615341588846]
We propose a rigorous estimation of the global curvature of weights across layers by approximating and controlling the norm of their Hessian matrix.
Our experiments on Word2Vec and the MNIST/CIFAR image classification tasks confirm that tracking the Hessian norm is a useful diagnostic tool.
arXiv Detail & Related papers (2020-04-20T18:12:56Z) - Widening and Squeezing: Towards Accurate and Efficient QNNs [125.172220129257]
Quantization neural networks (QNNs) are very attractive to the industry because their extremely cheap calculation and storage overhead, but their performance is still worse than that of networks with full-precision parameters.
Most of existing methods aim to enhance performance of QNNs especially binary neural networks by exploiting more effective training techniques.
We address this problem by projecting features in original full-precision networks to high-dimensional quantization features.
arXiv Detail & Related papers (2020-02-03T04:11:13Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.