Beyond Activation Alignment:The Alignment-Diversity Tradeoff in Task-Aware LLM Quantization

• URL: http://arxiv.org/abs/2607.00908v1
• Date: Wed, 01 Jul 2026 13:12:21 GMT
• Title: Beyond Activation Alignment:The Alignment-Diversity Tradeoff in Task-Aware LLM Quantization
• Abstract summary: We propose TASA (Task-Aware Sensitivity Analysis) to jointly optimize calibration-data composition and mixed-precision bit allocation.<n>Experiments on LLaMA-3-8B and Qwen2.5-7B reveal a precision inversion: appropriately allocated 3.5-bit models can match or surpass less task-aware 4-bit baselines.
• Score: 36.286359836094086
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Mixed-precision quantization (MPQ) has become a key technique for deploying large language models under stringent memory and compute constraints. We first identify a phenomenon that we term the Perplexity Illusion: layers ranked as important by perplexity-based sensitivity show little rank correlation with those that are most influential for complex reasoning performance, with Kendall $τ\approx 0$ in our analysis. We further reveal an Alignment-Diversity Tradeoff: using only target-task calibration data can degrade post-quantization performance, whereas incorporating general-domain data stabilizes sensitivity estimation and improves robustness across tasks. Based on these observations, we propose TASA (Task-Aware Sensitivity Analysis), a two-level framework that jointly optimizes calibration-data composition and mixed-precision bit allocation. Specifically, TASA searches for a calibration-data mixture using a training-free gradient-trace alignment criterion, and then aggregates perplexity and reasoning-oriented sensitivity signals to guide both inter-layer and intra-layer bit allocation. Experiments on LLaMA-3-8B and Qwen2.5-7B reveal a precision inversion: appropriately allocated 3.5-bit models can match or surpass less task-aware 4-bit baselines. At an average precision of 3.5 bits, TASA matches or outperforms several competitive 4-bit uniform baselines in aggregate accuracy, and improves over the strongest W3 baseline on GSM8K by more than 20 absolute points on LLaMA-3-8B. These results show that calibration-data composition substantially affects task-sensitive quantization, a factor underexplored in prior work.

Related papers

• MXSens: Sensitivity-Aware Mixed-Precision Quantization for Efficient LLM Inference [36.01709922688871]
  We introduce MXSens, a training-free method that assigns mixed mantissa bitwidths based on column- and layer-wise sensitivity.<n> MXSens achieves perplexities of 3.77 and 7.63 on LLaMA-2-70B and LLaMA-3-8B, respectively, substantially improving over existing baselines on WikiText-2.
  arXiv  Detail & Related papers  (2026-07-20T09:23:10Z)
• FoundationGeo: Learning Spatial Pixel-Wise Fields for Monocular Metric Geometry [52.232826445242644]
  FoundationGeo bridges relative and metric prediction via spatial calibration and principled data design.<n>Stage 1 learns a high-fidelity, affine-invariant geometry model by initializing with DINOv3.<n>Stage 2 moves beyond global scaling by introducing pixel-wise calibration fields for metric estimation.
  arXiv  Detail & Related papers  (2026-07-13T14:10:01Z)
• KPGrasp: Scalable Keypoint Flow Matching for Dexterous Grasp Generation [23.028638559051597]
  We present KPGrasp, a flow-matching framework that learns dexterous grasp priors from large-scale data.<n>Experiments on 20 diverse objects demonstrate that the pipeline can be deployed in a real-world setup.
  arXiv  Detail & Related papers  (2026-06-08T10:19:29Z)
• CAGE: Curvature-Aware Gradient Estimation For Accurate Quantization-Aware Training [73.46600457802693]
  We introduce a new method that counteracts the loss induced by quantization.<n>CAGE significantly improves upon the state-of-theart methods in terms of accuracy, for similar computational cost.<n>For QAT pre-training of Llama models, CAGE matches the accuracy achieved at 4-bits (W4A4) with the prior best method.
  arXiv  Detail & Related papers  (2025-10-21T16:33:57Z)
• Winning the Pruning Gamble: A Unified Approach to Joint Sample and Token Pruning for Efficient Supervised Fine-Tuning [71.30276778807068]
  We propose a unified framework that strategically coordinates sample pruning and token pruning.<n>Q-Tuning achieves a +38% average improvement over the full-data SFT baseline using only 12.5% of the original training data.
  arXiv  Detail & Related papers  (2025-09-28T13:27:38Z)
• IMPQ: Interaction-Aware Layerwise Mixed Precision Quantization for LLMs [4.655407920049974]
  Mixed-precision quantization offers a compelling solution, but existing methods struggle when the average precision drops below four bits.<n>In this paper, we propose two innovations to address these limitations.<n>First, we frame the mixed-precision quantization problem as a cooperative game among layers and introduce Shapley-based Progressive Quantization Estimation (SPQE)<n>Second, we propose Interaction-aware Mixed-Precision Quantization (IMPQ) which translates these Shapley estimates into a binary quadratic optimization formulation.
  arXiv  Detail & Related papers  (2025-09-18T21:59:40Z)
• Pushing the Limits of Low-Bit Optimizers: A Focus on EMA Dynamics [64.62231094774211]
  Statefuls (e.g., Adam) maintain auxiliary information even 2x the model size in order to achieve optimal convergence.<n>SOLO enables Adam-styles to maintain quantized states with precision as low as 3 bits, or even 2 bits.<n>SOLO can thus be seamlessly applied to Adam-styles, leading to substantial memory savings with minimal accuracy loss.
  arXiv  Detail & Related papers  (2025-05-01T06:47:45Z)
• QUAD: Quantization and Parameter-Efficient Tuning of LLM with Activation Decomposition [21.13478769431063]
  QUAD (Quantization with Activation Decomposition) is a framework leveraging Singular Value Decomposition (SVD) to suppress activation outliers for effective 4-bit quantization.<n>We show QUAD achieves 94% 96% accuracy under W4A4 quantization and 98% accuracy with W4A4/A8 and parameter-efficient fine-tuning for Llama-3 and Qwen-2.5 models.
  arXiv  Detail & Related papers  (2025-03-25T05:03:56Z)
• GAQAT: gradient-adaptive quantization-aware training for domain generalization [54.31450550793485]
  We propose a novel Gradient-Adaptive Quantization-Aware Training (GAQAT) framework for DG.<n>Our approach begins by identifying the scale-gradient conflict problem in low-precision quantization.<n>Extensive experiments validate the effectiveness of the proposed GAQAT framework.
  arXiv  Detail & Related papers  (2024-12-07T06:07:21Z)
• On Calibration of Modern Quantized Efficient Neural Networks [79.06893963657335]
  Quality of calibration is observed to track the quantization quality. GhostNet-VGG is shown to be the most robust to overall performance drop at lower precision.
  arXiv  Detail & Related papers  (2023-09-25T04:30:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.