Adaptive Layer-Wise Transformations for Post-Training Quantization of Large Language Models

• URL: http://arxiv.org/abs/2511.17809v1
• Date: Fri, 21 Nov 2025 22:01:58 GMT
• Title: Adaptive Layer-Wise Transformations for Post-Training Quantization of Large Language Models
• Authors: Cuong Pham, Hoang Anh Dung, Cuong C. Nguyen, Trung Le, Gustavo Carneiro, Jianfei Cai, Thanh-Toan Do,
• Abstract summary: Large language models require significant computational resources for deployment.<n>Main obstacle to effective quantization lies in systematic outliers in activations and weights.<n>We propose an adaptive transformation selection framework that systematically determines optimal transformations on a per-layer basis.
• Score: 47.54958360970588
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models require significant computational resources for deployment, making quantization essential for practical applications. However, the main obstacle to effective quantization lies in systematic outliers in activations and weights, which cause substantial LLM performance degradation, especially at low-bit settings. While existing transformation-based methods like affine and rotation transformations successfully mitigate outliers, they apply the homogeneous transformation setting, i.e., using the same transformation types across all layers, ignoring the heterogeneous distribution characteristics within LLMs. In this paper, we propose an adaptive transformation selection framework that systematically determines optimal transformations on a per-layer basis. To this end, we first formulate transformation selection as a differentiable optimization problem to achieve the accurate transformation type for each layer. However, searching for optimal layer-wise transformations for every model is computationally expensive. To this end, we establish the connection between weight distribution kurtosis and accurate transformation type. Specifically, we propose an outlier-guided layer selection method using robust $z$-score normalization that achieves comparable performance to differentiable search with significantly reduced overhead. Comprehensive experiments on LLaMA family models demonstrate that our adaptive approach consistently outperforms the widely-used fixed transformation settings. For example, our method achieves an improvement of up to 4.58 perplexity points and a 2.11% gain in average six-task zero-shot accuracy under aggressive W3A3K2V2 quantization settings for the LLaMA-3-8B model compared to the current best existing method, FlatQuant, demonstrating the necessity of heterogeneous transformation selection for optimal LLM quantization.

Related papers

• Towards Robust Scaling Laws for Optimizers [89.21160945066737]
  Empirical scaling laws are widely used to predict loss as model size and training data grow.<n>We show that Chinchilla-style scaling laws emerge naturally as a result of loss decomposition into irreducible, approximation, and optimization errors.
  arXiv  Detail & Related papers  (2026-02-07T21:40:33Z)
• Variational Entropic Optimal Transport [67.76725267984578]
  We propose Variational Entropic Optimal Transport (VarEOT) for domain translation problems.<n>VarEOT is based on an exact variational reformulation of the log-partition $log mathbbE[exp(cdot)$ as a tractable generalization over an auxiliary positive normalizer.<n> Experiments on synthetic data and unpaired image-to-image translation demonstrate competitive or improved translation quality.
  arXiv  Detail & Related papers  (2026-02-02T15:48:44Z)
• WUSH: Near-Optimal Adaptive Transforms for LLM Quantization [52.77441224845925]
  Quantization to low bitwidth is a standard approach for deploying large language models.<n>A few extreme weights and activations stretch the dynamic range and reduce the effective resolution of the quantizer.<n>We derive, for the first time, closed-form optimal linear blockwise transforms for joint weight-activation quantization.
  arXiv  Detail & Related papers  (2025-11-30T16:17:34Z)
• Taming LLMs by Scaling Learning Rates with Gradient Grouping [49.91587150497186]
  Training large language models (LLMs) poses challenges due to their massive scale and heterogeneous architectures.<n>This work introduces Scaling with Gradient Grouping (SGG), an gradient wrapper that improves adaptive learning rate estimation by dynamic grouping and group-specific scaling.
  arXiv  Detail & Related papers  (2025-06-01T15:30:37Z)
• Efficient Shapley Value-based Non-Uniform Pruning of Large Language Models [43.4962029013024]
  Pruning large language models (LLMs) is a promising solution for reducing model sizes and computational complexity while preserving performance.<n>We propose the Shapley Value-based Non-Uniform Pruning (SV-NUP) method for LLMs.<n>This approach quantifies the contribution of each transformer layer to the overall model performance, enabling the assignment of tailored pruning budgets to different layers to retain critical parameters.
  arXiv  Detail & Related papers  (2025-05-03T07:57:02Z)
• Beyond One-Size-Fits-All Pruning via Evolutionary Metric Search for Large Language Models [18.57876883968734]
  We introduce textbftextscOptiShear, an efficient evolutionary optimization framework for adaptive LLM pruning.<n>Our framework features two key innovations: an effective search space built on our Meta pruning metric, and a model-wise reconstruction error for rapid evaluation.
  arXiv  Detail & Related papers  (2025-02-15T09:17:38Z)
• AffineQuant: Affine Transformation Quantization for Large Language Models [58.45460102764]
  Post-Training Quantization (PTQ) has emerged as a subject of considerable interest due to its compression efficiency and cost-effectiveness in the context of training. Existing PTQ methods for Large-scale Language Models (LLMs) limit the optimization scope to scaling transformations between pre- and post-quantization weights. In this paper, we advocate for the direct optimization using equivalent Affine transformations in PTQ (AffineQuant)
  arXiv  Detail & Related papers  (2024-03-19T08:40:21Z)
• Studying Evolutionary Solution Adaption Using a Flexibility Benchmark Based on a Metal Cutting Process [39.05320053926048]
  We consider optimizing for different production requirements from the viewpoint of a bio-inspired framework for system flexibility.<n>We study the flexibility of NSGA-II, which we extend by two variants: 1) varying goals, which optimize solutions for two tasks simultaneously to obtain in-between source solutions expected to be more adaptable, and 2) active-inactive genotype, which accommodates different possibilities that can be activated or deactivated.
  arXiv  Detail & Related papers  (2023-05-31T12:07:50Z)
• Resource-Efficient Invariant Networks: Exponential Gains by Unrolled Optimization [8.37077056358265]
  We propose a new computational primitive for building invariant networks based instead on optimization. We provide empirical and theoretical corroboration of the efficiency gains and soundness of our proposed method. We demonstrate its utility in constructing an efficient invariant network for a simple hierarchical object detection task.
  arXiv  Detail & Related papers  (2022-03-09T19:04:08Z)
• Meta-Learning with Neural Tangent Kernels [58.06951624702086]
  We propose the first meta-learning paradigm in the Reproducing Kernel Hilbert Space (RKHS) induced by the meta-model's Neural Tangent Kernel (NTK) Within this paradigm, we introduce two meta-learning algorithms, which no longer need a sub-optimal iterative inner-loop adaptation as in the MAML framework. We achieve this goal by 1) replacing the adaptation with a fast-adaptive regularizer in the RKHS; and 2) solving the adaptation analytically based on the NTK theory.
  arXiv  Detail & Related papers  (2021-02-07T20:53:23Z)
• Efficient Semantic Image Synthesis via Class-Adaptive Normalization [116.63715955932174]
  Class-adaptive normalization (CLADE) is a lightweight but equally-effective variant that is only adaptive to semantic class. We introduce intra-class positional map encoding calculated from semantic layouts to modulate the normalization parameters of CLADE. The proposed CLADE can be generalized to different SPADE-based methods while achieving comparable generation quality compared to SPADE.
  arXiv  Detail & Related papers  (2020-12-08T18:59:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.