Hard Sample Matters a Lot in Zero-Shot Quantization
- URL: http://arxiv.org/abs/2303.13826v1
- Date: Fri, 24 Mar 2023 06:22:57 GMT
- Title: Hard Sample Matters a Lot in Zero-Shot Quantization
- Authors: Huantong Li, Xiangmiao Wu, Fanbing Lv, Daihai Liao, Thomas H. Li,
Yonggang Zhang, Bo Han, Mingkui Tan
- Abstract summary: Zero-shot quantization (ZSQ) is promising for compressing and accelerating deep neural networks when the data for training full-precision models are inaccessible.
In ZSQ, network quantization is performed using synthetic samples, thus, the performance of quantized models depends heavily on the quality of synthetic samples.
We propose HArd sample Synthesizing and Training (HAST) to address this issue.
- Score: 52.32914196337281
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Zero-shot quantization (ZSQ) is promising for compressing and accelerating
deep neural networks when the data for training full-precision models are
inaccessible. In ZSQ, network quantization is performed using synthetic
samples, thus, the performance of quantized models depends heavily on the
quality of synthetic samples. Nonetheless, we find that the synthetic samples
constructed in existing ZSQ methods can be easily fitted by models.
Accordingly, quantized models obtained by these methods suffer from significant
performance degradation on hard samples. To address this issue, we propose HArd
sample Synthesizing and Training (HAST). Specifically, HAST pays more attention
to hard samples when synthesizing samples and makes synthetic samples hard to
fit when training quantized models. HAST aligns features extracted by
full-precision and quantized models to ensure the similarity between features
extracted by these two models. Extensive experiments show that HAST
significantly outperforms existing ZSQ methods, achieving performance
comparable to models that are quantized with real data.
Related papers
- Provable Statistical Rates for Consistency Diffusion Models [87.28777947976573]
Despite the state-of-the-art performance, diffusion models are known for their slow sample generation due to the extensive number of steps involved.
This paper contributes towards the first statistical theory for consistency models, formulating their training as a distribution discrepancy minimization problem.
arXiv Detail & Related papers (2024-06-23T20:34:18Z) - Enhanced Distribution Alignment for Post-Training Quantization of
Diffusion Models [4.601488148143309]
Quantization can effectively reduce model complexity, and post-training quantization is highly promising in accelerating the denoising process.
Existing PTQ methods for diffusion models suffer from distribution mismatch issues at both calibration sample level and reconstruction output level.
We propose Enhanced Distribution Alignment for Post-Training Quantization of Diffusion Models (EDA-DM) to address the above issues.
EDA-DM outperforms the existing post-training quantization frameworks in both unconditional and conditional generation scenarios.
arXiv Detail & Related papers (2024-01-09T14:42:49Z) - Importance sampling for stochastic quantum simulations [68.8204255655161]
We introduce the qDrift protocol, which builds random product formulas by sampling from the Hamiltonian according to the coefficients.
We show that the simulation cost can be reduced while achieving the same accuracy, by considering the individual simulation cost during the sampling stage.
Results are confirmed by numerical simulations performed on a lattice nuclear effective field theory.
arXiv Detail & Related papers (2022-12-12T15:06:32Z) - Mixed-Precision Inference Quantization: Radically Towards Faster
inference speed, Lower Storage requirement, and Lower Loss [4.877532217193618]
Existing quantization techniques rely heavily on experience and "fine-tuning" skills.
This study provides a methodology for acquiring a mixed-precise quantization model with a lower loss than the full precision model.
In particular, we will demonstrate that neural networks with massive identity mappings are resistant to the quantization method.
arXiv Detail & Related papers (2022-07-20T10:55:34Z) - ClusterQ: Semantic Feature Distribution Alignment for Data-Free
Quantization [111.12063632743013]
We propose a new and effective data-free quantization method termed ClusterQ.
To obtain high inter-class separability of semantic features, we cluster and align the feature distribution statistics.
We also incorporate the intra-class variance to solve class-wise mode collapse.
arXiv Detail & Related papers (2022-04-30T06:58:56Z) - Diverse Sample Generation: Pushing the Limit of Data-free Quantization [85.95032037447454]
This paper presents a generic Diverse Sample Generation scheme for the generative data-free post-training quantization and quantization-aware training.
For large-scale image classification tasks, our DSG can consistently outperform existing data-free quantization methods.
arXiv Detail & Related papers (2021-09-01T07:06:44Z) - Closed-form Continuous-Depth Models [99.40335716948101]
Continuous-depth neural models rely on advanced numerical differential equation solvers.
We present a new family of models, termed Closed-form Continuous-depth (CfC) networks, that are simple to describe and at least one order of magnitude faster.
arXiv Detail & Related papers (2021-06-25T22:08:51Z) - Exponential Reduction in Sample Complexity with Learning of Ising Model
Dynamics [14.704630929165274]
We study the problem of reconstructing binary graphical models from correlated samples produced by a dynamical process.
We analyze the sample complexity of two estimators that are based on the interaction screening objective and the conditional likelihood loss.
arXiv Detail & Related papers (2021-04-02T11:44:13Z) - Zero-shot Adversarial Quantization [11.722728148523366]
We propose a zero-shot adversarial quantization (ZAQ) framework, facilitating effective discrepancy estimation and knowledge transfer.
This is achieved by a novel two-level discrepancy modeling to drive a generator to synthesize informative and diverse data examples.
We conduct extensive experiments on three fundamental vision tasks, demonstrating the superiority of ZAQ over the strong zero-shot baselines.
arXiv Detail & Related papers (2021-03-29T01:33:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.