Amortized-Precision Quantization for Early-Exit Vision Transformers

• URL: http://arxiv.org/abs/2605.07317v1
• Date: Fri, 08 May 2026 06:27:04 GMT
• Title: Amortized-Precision Quantization for Early-Exit Vision Transformers
• Abstract summary: We introduce Amortized-Precision Quantization (APQ), a utilization-aware formulation that accounts for layer-wise exposure to quantization noise.<n>We propose Mutual Adaptive Quantization with Early Exiting (MAQEE), a bi-level framework that jointly optimize exit thresholds and bit-widths under explicit risk control to improve inference stability.
• Score: 14.001192614503807
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Vision Transformers (ViTs) achieve strong performance across vision tasks, yet their deployment with low-precision early exiting remains fragile. Existing quantization methods assume static full-depth execution, making them unstable when exit decisions are perturbed by quantization noise, which can amplify errors along dynamic inference paths. In this paper, we introduce Amortized-Precision Quantization (APQ), a utilization-aware formulation that accounts for layer-wise stochastic exposure to quantization noise and reveals depth-precision trade-offs. Building on APQ, we propose Mutual Adaptive Quantization with Early Exiting (MAQEE), a bi-level framework that jointly optimizes exit thresholds and bit-widths under explicit risk control to improve inference stability. MAQEE establishes a superior Pareto frontier in the accuracy-efficiency trade-off, reducing BOPs by up to 95% while maintaining accuracy and outperforming strong baselines by up to 20\% across classification, detection, and segmentation tasks.

Related papers

• Distribution-Aware Robust Bilevel Optimization: Quantile-Guided Huber Updates in Two-Timescale Stochastic Approximation [16.106678611834102]
  Bilevel optimization (BLO) is fundamental to hierarchical decision-making but suffers from critical instability under heavy-tailed noise.<n>Existing techniques typically rely on magnitude checks, which fail to distinguish informative geometric signals from outliers.
  arXiv  Detail & Related papers  (2026-06-21T10:57:37Z)
• DiRecT: Safe Diffusion-Based Planning via Receding-Horizon Denoising [4.249024052507976]
  We introduce DiRecT, a training-free algorithm for constrained sampling from diffusion models via optimal control (SOC)<n>Inspired by model predictive control, we derive a receding-horizon surrogate for the otherwise intractable constrained SOC formulation.<n>Experiments demonstrate that DiRecT substantially improves deployment safety and task performance over existing diffusion-based planning baselines.
  arXiv  Detail & Related papers  (2026-06-13T15:41:26Z)
• Decision-calibrated prediction sets for robust power system operations [2.234083934481286]
  We introduce decisioncalibrated uncertainty sets to balance operating costs and reliability in power systems dominated by weather-dependent uncertainty.<n>These are defined in terms of the reliability of downstream decisions, rather than in terms of conditional coverage.<n>We show that decision-calibrated sets attain prescribed constraint-satisfaction targets within about three percentage points, whereas standard coverage-based calibration systematically exceeds these targets by more than eleven percentage points.
  arXiv  Detail & Related papers  (2026-06-01T11:12:09Z)
• ECO: Quantized Training without Full-Precision Master Weights [58.97082407934466]
  Error-Compensating (ECO) eliminates master weights by applying updates directly to quantized parameters.<n>We show that ECO converges to a constant-radius neighborhood of the optimum, while naive master-weight removal can incur an error that is inversely proportional to the learning rate.
  arXiv  Detail & Related papers  (2026-01-29T18:35:01Z)
• CAGE: Curvature-Aware Gradient Estimation For Accurate Quantization-Aware Training [73.46600457802693]
  We introduce a new method that counteracts the loss induced by quantization.<n>CAGE significantly improves upon the state-of-theart methods in terms of accuracy, for similar computational cost.<n>For QAT pre-training of Llama models, CAGE matches the accuracy achieved at 4-bits (W4A4) with the prior best method.
  arXiv  Detail & Related papers  (2025-10-21T16:33:57Z)
• Beyond Outliers: A Study of Optimizers Under Quantization [82.75879062804955]
  We study impact of choice on model robustness under quantization.<n>We evaluate how model performance degrades when trained with different baselines.<n>We derive scaling laws for quantization-aware training under different parameters.
  arXiv  Detail & Related papers  (2025-09-27T21:15:22Z)
• Progressive Element-wise Gradient Estimation for Neural Network Quantization [2.1413624861650358]
  Quantization-Aware Training (QAT) methods rely on the Straight-Through Estimator (STE) to address the non-differentiability of discretization functions.<n>We propose Progressive Element-wise Gradient Estimation (PEGE) to address discretization errors between continuous and quantized values.<n>PEGE consistently outperforms existing backpropagation methods and enables low-precision models to match or even outperform the accuracy of their full-precision counterparts.
  arXiv  Detail & Related papers  (2025-08-27T15:59:36Z)
• End-to-End On-Device Quantization-Aware Training for LLMs at Inference Cost [53.25965863436039]
  Quantization-aware training (QAT) provides a more principled solution, but its reliance on backpropagation incurs prohibitive memory costs.<n>We propose ZeroQAT, a zeroth-order optimization-based QAT framework that supports both weight and activation quantization.<n>Experiments show that ZeroQAT consistently outperforms representative PTQ and QAT baselines while requiring significantly less memory.
  arXiv  Detail & Related papers  (2025-08-21T01:18:27Z)
• MPQ-DMv2: Flexible Residual Mixed Precision Quantization for Low-Bit Diffusion Models with Temporal Distillation [74.34220141721231]
  We present MPQ-DMv2, an improved textbfMixed textbfPrecision textbfQuantization framework for extremely low-bit textbfDiffusion textbfModels.
  arXiv  Detail & Related papers  (2025-07-06T08:16:50Z)
• Token Constraint Decoding Improves Robustness on Question Answering for Large Language Models [4.078176555898098]
  We introduce and evaluate Token Constraint Decoding (TCD)<n>This simple yet effective inference-time algorithm enforces alignment between token-level predictions to enhance robustness in noisy settings.<n>Our findings establish TCD as a practical, model-agnostic approach for improving reasoning stability under real-world imperfections.
  arXiv  Detail & Related papers  (2025-06-11T05:33:56Z)
• SeWA: Selective Weight Average via Probabilistic Masking [51.015724517293236]
  We show that only a few points are needed to achieve better and faster convergence.<n>We transform the discrete selection problem into a continuous subset optimization framework.<n>We derive the SeWA's stability bounds, which are sharper than that under both convex image checkpoints.
  arXiv  Detail & Related papers  (2025-02-14T12:35:21Z)
• DopQ-ViT: Towards Distribution-Friendly and Outlier-Aware Post-Training Quantization for Vision Transformers [31.791935689364866]
  We propose DopQ-ViT, a Distribution-friendly and Outlier-aware Post-training Quantization method for Vision Transformers (ViTs)<n>First, DopQ-ViT introduces the Tan Quantizer (TanQ), which better preserves the power-law distribution of post-Softmax activations by focusing more on values near 1.<n>Second, DopQ-ViT presents the MAD-guided Optimal Scaling Factor (MOSF), which selects the optimal scaling factor without introducing additional calculations.
  arXiv  Detail & Related papers  (2024-08-06T16:40:04Z)
• Towards Accurate Post-Training Quantization for Vision Transformer [48.779346466374406]
  Existing post-training quantization methods still cause severe performance drops. APQ-ViT surpasses the existing post-training quantization methods by convincing margins.
  arXiv  Detail & Related papers  (2023-03-25T03:05:26Z)
• Neural Networks with Quantization Constraints [111.42313650830248]
  We present a constrained learning approach to quantization training. We show that the resulting problem is strongly dual and does away with gradient estimations. We demonstrate that the proposed approach exhibits competitive performance in image classification tasks.
  arXiv  Detail & Related papers  (2022-10-27T17:12:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.