IPTQ-ViT: Post-Training Quantization of Non-linear Functions for Integer-only Vision Transformers

• URL: http://arxiv.org/abs/2511.15369v1
• Date: Wed, 19 Nov 2025 11:56:16 GMT
• Title: IPTQ-ViT: Post-Training Quantization of Non-linear Functions for Integer-only Vision Transformers
• Abstract summary: Quantization-Aware Training (QAT) methods for vision transformers rely on expensive retraining to recover accuracy loss in non-linear layer quantization.<n>Existing Post-Training Quantization (PTQ) methods either partially quantize non-linear functions or adjust activation distributions to maintain accuracy but fail to achieve fully integer-only inference.<n>We introduce IPTQ-ViT, a novel PTQ framework for fully integer-only vision transformers without retraining.
• Score: 15.498250598583484
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Previous Quantization-Aware Training (QAT) methods for vision transformers rely on expensive retraining to recover accuracy loss in non-linear layer quantization, limiting their use in resource-constrained environments. In contrast, existing Post-Training Quantization (PTQ) methods either partially quantize non-linear functions or adjust activation distributions to maintain accuracy but fail to achieve fully integer-only inference. In this paper, we introduce IPTQ-ViT, a novel PTQ framework for fully integer-only vision transformers without retraining. We present approximation functions: a polynomial-based GELU optimized for vision data and a bit-shifting-based Softmax designed to improve approximation accuracy in PTQ. In addition, we propose a unified metric integrating quantization sensitivity, perturbation, and computational cost to select the optimal approximation function per activation layer. IPTQ-ViT outperforms previous PTQ methods, achieving up to 6.44\%p (avg. 1.78\%p) top-1 accuracy improvement for image classification, 1.0 mAP for object detection. IPTQ-ViT outperforms partial floating-point PTQ methods under W8A8 and W4A8, and achieves accuracy and latency comparable to integer-only QAT methods. We plan to release our code https://github.com/gihwan-kim/IPTQ-ViT.git.

Related papers

• Efficient Tuning Before Low-Bit Post-Training Quantization for Stochastic Gradient Descent-optimized Models [8.779009700879982]
  Post-training quantization (PTQ) compresses deep neural networks for deployment under limited memory and computational budgets.<n>Most existing PTQ methods operate on an unconstrained full-precision (FP) model and primarily address quantization errors through post-hoc reconstruction.<n>We argue that low-bit PTQ accuracy is limited not only by post-quantization error minimization, but also by the quantization-error tolerance of FP model itself.
  arXiv  Detail & Related papers  (2026-07-13T10:24:36Z)
• ScalePredictor: Instance-aware Scale Learning for Accurate Quantization of Vision Transformers [60.09590321091875]
  Post-Training Quantization (PTQ) offers an attractive solution by compressing models using a small calibration set with minimal training overhead.<n>Most existing PTQ works adopt a static quantization paradigm that is uniformly applied to all instances.<n>We propose ScalePredictor, a dynamic quantization framework for accurate and efficient quantization scale learning of ViTs.
  arXiv  Detail & Related papers  (2026-06-20T08:33:55Z)
• CAGE: Curvature-Aware Gradient Estimation For Accurate Quantization-Aware Training [73.46600457802693]
  We introduce a new method that counteracts the loss induced by quantization.<n>CAGE significantly improves upon the state-of-theart methods in terms of accuracy, for similar computational cost.<n>For QAT pre-training of Llama models, CAGE matches the accuracy achieved at 4-bits (W4A4) with the prior best method.
  arXiv  Detail & Related papers  (2025-10-21T16:33:57Z)
• Cat: Post-Training Quantization Error Reduction via Cluster-based Affine Transformation [47.791962198275066]
  Post-Training Quantization (PTQ) reduces the memory footprint and computational overhead of deep neural networks by converting full-precision (FP) values into quantized and compressed data types.<n>While PTQ is more cost-efficient than Quantization-Aware Training (QAT), it is highly susceptible to accuracy degradation under a low-bit quantization regime.<n>We propose Cluster-based Affine Transformation (CAT), an error-reduction framework that employs cluster-specific parameters to align LQ outputs with FP counterparts.
  arXiv  Detail & Related papers  (2025-09-30T14:00:28Z)
• Beyond Outliers: A Study of Optimizers Under Quantization [82.75879062804955]
  We study impact of choice on model robustness under quantization.<n>We evaluate how model performance degrades when trained with different baselines.<n>We derive scaling laws for quantization-aware training under different parameters.
  arXiv  Detail & Related papers  (2025-09-27T21:15:22Z)
• PTQTP: Post-Training Quantization to Trit-Planes for Large Language Models [29.616604431869746]
  Post-training quantization of large language models (LLMs) to extremely low bit-widths remains challenging.<n>Existing ultra-low-bit PTQ methods rely on binary approximations or complex compensation mechanisms.<n>We introduce PTQ to Trit-Planes (PTQTP), the first ternary-weight PTQ framework that decomposes weight matrices into structured ternary -1, 0, 1 trit-planes.
  arXiv  Detail & Related papers  (2025-09-21T09:07:20Z)
• End-to-End On-Device Quantization-Aware Training for LLMs at Inference Cost [53.25965863436039]
  Quantization-aware training (QAT) provides a more principled solution, but its reliance on backpropagation incurs prohibitive memory costs.<n>We propose ZeroQAT, a zeroth-order optimization-based QAT framework that supports both weight and activation quantization.<n>Experiments show that ZeroQAT consistently outperforms representative PTQ and QAT baselines while requiring significantly less memory.
  arXiv  Detail & Related papers  (2025-08-21T01:18:27Z)
• GPLQ: A General, Practical, and Lightning QAT Method for Vision Transformers [11.452135395287119]
  Vision Transformers (ViTs) are essential in computer vision but are computationally intensive, too.<n>Model quantization aims to alleviate this difficulty, yet existing Post-Training Quantization (PTQ) and Quantization-Aware Training (QAT) methods exhibit significant limitations.<n>This paper introduces General, Practical, and Quantization (GPLQ), a novel framework for efficient ViT quantization.
  arXiv  Detail & Related papers  (2025-06-13T13:45:17Z)
• FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximation [55.12070409045766]
  Post-training quantization (PTQ) has stood out as a cost-effective and promising model compression paradigm in recent years.<n>Current PTQ methods for Vision Transformers (ViTs) still suffer from significant accuracy degradation, especially under low-bit quantization.
  arXiv  Detail & Related papers  (2025-06-13T07:57:38Z)
• APHQ-ViT: Post-Training Quantization with Average Perturbation Hessian Based Reconstruction for Vision Transformers [71.2294205496784]
  We propose textbfAPHQ-ViT, a novel PTQ approach based on importance estimation with Average Perturbation Hessian (APH)<n>We show that APHQ-ViT using linear quantizers outperforms existing PTQ methods by substantial margins in 3-bit and 4-bit across different vision tasks.
  arXiv  Detail & Related papers  (2025-04-03T11:48:56Z)
• FP4DiT: Towards Effective Floating Point Quantization for Diffusion Transformers [15.324769026957641]
  Post-training quantization is a lightweight method to alleviate burdens without the need for training or fine-tuning.<n>We introduce FP4DiT, a PTQ method that leverages Floating-Point Quantization to achieve W4A6 quantization.<n> Experimental results demonstrate that FP4DiT outperforms integer-based PTQ at W4A6 and W4A8 precision.
  arXiv  Detail & Related papers  (2025-03-19T17:44:21Z)
• AdaLog: Post-Training Quantization for Vision Transformers with Adaptive Logarithm Quantizer [54.713778961605115]
  Vision Transformer (ViT) has become one of the most prevailing fundamental backbone networks in the computer vision community. We propose a novel non-uniform quantizer, dubbed the Adaptive Logarithm AdaLog (AdaLog) quantizer.
  arXiv  Detail & Related papers  (2024-07-17T18:38:48Z)
• Shedding the Bits: Pushing the Boundaries of Quantization with Minifloats on FPGAs [39.410068572891475]
  Post-training quantization (PTQ) is a powerful technique for model compression, reducing the numerical precision in neural networks without additional training overhead. Recent works have investigated adopting 8-bit floating-point formats(FP8) in the context of PTQ for model inference. We present minifloats, which are reduced-precision floating-point formats capable of further reducing the memory footprint, latency, and energy cost of a model.
  arXiv  Detail & Related papers  (2023-11-21T05:27:16Z)
• Gradient-Based Post-Training Quantization: Challenging the Status Quo [23.1120983784623]
  Quantization has become a crucial step for the efficient deployment of deep neural networks. In this work, we show that the process is, to a certain extent, robust to a number of variables. We derive a number of best practices for designing more efficient and scalable GPTQ methods.
  arXiv  Detail & Related papers  (2023-08-15T09:25:11Z)
• RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformers [2.114921680609289]
  We propose RepQ-ViT, a novel PTQ framework for vision transformers (ViTs) RepQ-ViT decouples the quantization and inference processes. It can outperform existing strong baselines and encouragingly improve the accuracy of 4-bit PTQ of ViTs to a usable level.
  arXiv  Detail & Related papers  (2022-12-16T02:52:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.