ISQuant: apply squant to the real deployment

• URL: http://arxiv.org/abs/2407.11037v1
• Date: Fri, 5 Jul 2024 15:10:05 GMT
• Title: ISQuant: apply squant to the real deployment
• Authors: Dezan Zhao,
• Abstract summary: We analyze why the combination of quantization and dequantization is used to train the model. We propose ISQuant as a solution for deploying 8-bit models.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The model quantization technique of deep neural networks has garnered significant attention and has proven to be highly useful in compressing model size, reducing computation costs, and accelerating inference. Many researchers employ fake quantization for analyzing or training the quantization process. However, fake quantization is not the final form for deployment, and there exists a gap between the academic setting and real-world deployment. Additionally, the inclusion of additional computation with scale and zero-point makes deployment a challenging task. In this study, we first analyze why the combination of quantization and dequantization is used to train the model and draw the conclusion that fake quantization research is reasonable due to the disappearance of weight gradients and the ability to approximate between fake and real quantization. Secondly, we propose ISQuant as a solution for deploying 8-bit models. ISQuant is fast and easy to use for most 8-bit models, requiring fewer parameters and less computation. ISQuant also inherits the advantages of SQuant, such as not requiring training data and being very fast at the first level of quantization. Finally We conduct some experiments and found the results is acceptable.our code is available at https://github.com/

Related papers

Err

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.