MARR: Module-Adaptive Residual Reconstruction for Low-Bit Post-Training Quantization

• URL: http://arxiv.org/abs/2605.17997v1
• Date: Mon, 18 May 2026 07:51:49 GMT
• Title: MARR: Module-Adaptive Residual Reconstruction for Low-Bit Post-Training Quantization
• Abstract summary: We propose Module-Adaptive Residual Reconstruction (MARR)<n>MARR assigns a module-specific scaling coefficient to adaptively balance accumulated-error correction and residual-related HA bias for each module.<n>Experiments on several typical large language models (LLMs) and vision transformers (ViTs) demonstrate the effectiveness of MARR under low-bit quantization.
• Score: 42.904834974723535
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recently, residual reconstruction-based model quantization methods have achieved promising performance in low-bit post-training quantization (PTQ) by introducing cross-layer residuals to reduce error accumulated from previous layers.However, these residuals may also introduce additional bias arising from the Hessian-approximation (HA) assumption underlying reconstruction-based PTQ, leading to suboptimal quantization performance.In this work, we analyze that multiplying the residual term by a scaling coefficient provides a direct way to mitigate the HA bias associated with residual strength, while preserving accumulated-error correction. More importantly, we observe that this trade-off is module-dependent, making a single global residual strength insufficient to balance effective correction and residual-related bias across modules.Based on these observations, we propose Module-Adaptive Residual Reconstruction (MARR), which assigns a module-specific scaling coefficient to adaptively balance accumulated-error correction and residual-related HA bias for each module.To avoid expensive per-module coefficient search and obtain a stable coefficient estimate, we design a Proportional-Integral-Derivative (PID)-based adaptive update strategy that uses reconstruction error as feedback to progressively refine this coefficient. Experiments on several typical large language models (LLMs) and vision transformers (ViTs) demonstrate the effectiveness of MARR under low-bit quantization (less than or equal to 4-bit), achieving up to 20.2% performance gains on LLMs and up to 4.6% relative gains on ViTs over the residual reconstruction state-of-the-art methods.Code will be made publicly available upon acceptance.

Related papers

• Reducing Learner Redundancy in Boosting via Residual Orthogonalization [18.700465769971434]
  Our framework tackles redundancy through two complementary mechanisms: Spectral Residual Projection (SRP) and Covariance-Regularized Weighting (CRW)<n>Experiments across ten benchmark datasets demonstrate that SCBoost delivers strong out-of-the-box performance.<n>This work reinterprets boosting through a geometric lens, suggesting that explicit redundancy control is a principled and necessary step toward more efficient ensemble architectures.
  arXiv  Detail & Related papers  (2026-06-16T06:13:59Z)
• Residual-Controlled Multiplier Learning for Stochastic Constrained Decision-Making [12.04727482907937]
  CML addresses the problem of tracking the noise of primal feedback multipliers under mini-batch conditions.<n>We decompose the multiplier into an effective pressure signal for primal descent and a finite-gain multiplier.
  arXiv  Detail & Related papers  (2026-06-05T09:35:22Z)
• LFQ: Logit-aware Final-block Quantization for Boosting the Generation Quality of Low-Bit Quantized LLMs [52.1276403258812]
  We introduce Logit-aware Final-block Quantization (LFQ), a simple yet effective enhancement to block-wise PTQ.<n>LFQ consistently improves the accuracy of complex generation tasks over state-of-the-art block-wise PTQ.
  arXiv  Detail & Related papers  (2026-05-28T11:02:23Z)
• DegBins: Degradation-Driven Binning for Depth Super-Resolution [57.61783461543096]
  DegBins is a novel DSR framework that leverages degradation-driven binning to adaptively enhance residual modeling.<n>DegBins consistently outperforms existing state-of-the-art methods in terms of accuracy, robustness, and generalization.
  arXiv  Detail & Related papers  (2026-05-10T16:17:11Z)
• Rethinking Residual Errors in Compensation-based LLM Quantization [15.416446372209924]
  We introduce an asymmetric calibration process that aligns the output of each quantized layer with its full-precision counterpart.<n>We reveal that the residual error originates not only from the output difference of the preceding layer but also from the discrepancy between the compensated and original weights within each layer.<n>Our proposed enhancements integrate seamlessly with both GPTQ and GPTAQ, significantly improving their quantization performance.
  arXiv  Detail & Related papers  (2026-04-09T08:20:59Z)
• LLMs can Compress LLMs: Adaptive Pruning by Agents [0.0]
  Post-training pruning has emerged as a promising approach to reduce computational costs while preserving performance.<n>We introduce agent-guided pruning, where a foundation model acts as an adaptive pruning agent.<n>We evaluate our approach on Q3 models (4B and 8B parameters) at approximately 45% sparsity, demonstrating substantial improvements over structured pruning baselines.
  arXiv  Detail & Related papers  (2026-01-14T18:45:36Z)
• End-to-End On-Device Quantization-Aware Training for LLMs at Inference Cost [53.25965863436039]
  Quantization-aware training (QAT) provides a more principled solution, but its reliance on backpropagation incurs prohibitive memory costs.<n>We propose ZeroQAT, a zeroth-order optimization-based QAT framework that supports both weight and activation quantization.<n>Experiments show that ZeroQAT consistently outperforms representative PTQ and QAT baselines while requiring significantly less memory.
  arXiv  Detail & Related papers  (2025-08-21T01:18:27Z)
• A Cycle-Consistency Constrained Framework for Dynamic Solution Space Reduction in Noninjective Regression [4.04042026249306]
  This paper proposes a cycle consistency-based data-driven training framework.<n>Experiments on normalized synthetic and simulated datasets demonstrate that the proposed method achieves a cycle reconstruction error below 0.003.<n>The framework significantly reduces reliance on manual intervention, demonstrating potential advantages in non-injective regression tasks.
  arXiv  Detail & Related papers  (2025-07-07T04:28:01Z)
• Self-Boost via Optimal Retraining: An Analysis via Approximate Message Passing [58.52119063742121]
  Retraining a model using its own predictions together with the original, potentially noisy labels is a well-known strategy for improving the model performance.<n>This paper addresses the question of how to optimally combine the model's predictions and the provided labels.<n>Our main contribution is the derivation of the Bayes optimal aggregator function to combine the current model's predictions and the given labels.
  arXiv  Detail & Related papers  (2025-05-21T07:16:44Z)
• APHQ-ViT: Post-Training Quantization with Average Perturbation Hessian Based Reconstruction for Vision Transformers [71.2294205496784]
  We propose textbfAPHQ-ViT, a novel PTQ approach based on importance estimation with Average Perturbation Hessian (APH)<n>We show that APHQ-ViT using linear quantizers outperforms existing PTQ methods by substantial margins in 3-bit and 4-bit across different vision tasks.
  arXiv  Detail & Related papers  (2025-04-03T11:48:56Z)
• Efficient Diffusion as Low Light Enhancer [63.789138528062225]
  Reflectance-Aware Trajectory Refinement (RATR) is a simple yet effective module to refine the teacher trajectory using the reflectance component of images. textbfReflectance-aware textbfDiffusion with textbfDistilled textbfTrajectory (textbfReDDiT) is an efficient and flexible distillation framework tailored for Low-Light Image Enhancement (LLIE)
  arXiv  Detail & Related papers  (2024-10-16T08:07:18Z)
• CBQ: Cross-Block Quantization for Large Language Models [66.82132832702895]
  Post-training quantization (PTQ) has played a key role in compressing large language models (LLMs) with ultra-low costs.<n>We propose CBQ, a cross-block reconstruction-based PTQ method for LLMs.<n> CBQ employs a cross-block dependency using a reconstruction scheme, establishing long-range dependencies across multiple blocks to minimize error accumulation.
  arXiv  Detail & Related papers  (2023-12-13T07:56:27Z)
• Probabilistic Traffic Forecasting with Dynamic Regression [15.31488551912888]
  This paper proposes a dynamic regression (DR) framework that enhances existing deeptemporal models by incorporating for learning the error process in traffic forecasting.<n>The framework relaxes the assumption of time independence by modeling the error series of the base model using a matrix- structured autoregressive (AR) model.<n>The newly designed loss function is based on the likelihood of a non-isotropic error term, enabling the model to generate probabilistic forecasts while preserving the original outputs of the base model.
  arXiv  Detail & Related papers  (2023-01-17T01:12:44Z)
• Modal Regression based Structured Low-rank Matrix Recovery for Multi-view Learning [70.57193072829288]
  Low-rank Multi-view Subspace Learning has shown great potential in cross-view classification in recent years. Existing LMvSL based methods are incapable of well handling view discrepancy and discriminancy simultaneously. We propose Structured Low-rank Matrix Recovery (SLMR), a unique method of effectively removing view discrepancy and improving discriminancy.
  arXiv  Detail & Related papers  (2020-03-22T03:57:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.