Silent Failures in Quantized LLM Reasoning: A Taxonomy-Based Analysis of Hollow Convergence and Failure Mode Shifts

• URL: http://arxiv.org/abs/2607.09999v1
• Date: Fri, 10 Jul 2026 21:55:05 GMT
• Title: Silent Failures in Quantized LLM Reasoning: A Taxonomy-Based Analysis of Hollow Convergence and Failure Mode Shifts
• Abstract summary: We show that post-training quantization can silently alter how large language models reason even when task accuracy is preserved.<n>We classify 30,000 chain-of-thought outputs from five instruction-tuned LLMs across three quantization precisions.<n>We find that while accuracy is robust across precisions, Hollow Convergence shows a significant size-dependent shift under NF4.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We show that post-training quantization can silently alter how large language models reason even when task accuracy is preserved. Using a six-category failure taxonomy validated by two independent human annotators (Cohen's $κ$ = 0.906), we classify 30,000 chain-of-thought outputs from five instruction-tuned LLMs (3B--14B parameters) across three quantization precisions (FP32, FP16, NF4) and four reasoning benchmarks. We find that while accuracy is robust across precisions (maximum 3.1 pp drop), Hollow Convergence (correct answers reached through incomplete or unverifiable reasoning) shows a significant size-dependent shift under NF4, dropping sharply for the two smallest models tested but remaining invariant for models at 12B parameters and above. This effect is also benchmark-specific: GSM8K is categorically immune while LogiQA and ARC-Challenge show the largest shifts. Furthermore, under NF4, Shortcut Collapse rises from 44% to 78% of wrong-answer failures in LLaMA 3.2-3B while Confidence Snowballing collapses from 15.8% to near zero, a qualitative shift invisible to accuracy metrics. Finally, we show Hollow Convergence cannot be reliably detected from surface-level text features (best F1 = 0.53), establishing it as a deployment-relevant failure mode that standard evaluation pipelines cannot catch.

Related papers

• Steering Under Compression: Dose-Response, Capability Cost, and Failure Asymmetry in Quantized LLMs [0.0]
  Inference-time steering enables behavioral control of large language models without parameter modification.<n>Post-training quantization reduces memory and compute costs for deployment.<n>We systematically study activation steering under weight-only quantization.
  arXiv  Detail & Related papers  (2026-09-06T08:41:42Z)
• Conformalized Large Language Models under Configuration Shift [46.502741608809174]
  Conformal prediction is a distribution-free framework for uncertainty quantification.<n>Nonconformity scores are often induced by an inference pipeline, not just a fixed model.<n>We show that configuration shift consistently erodes CP validity.
  arXiv  Detail & Related papers  (2026-08-02T19:34:41Z)
• Reliability Scaling Laws for Quantized Large Language Models [68.39028482878265]
  Quantization is a strategy to build capable and resource-efficient large language models (LLMs) by reducing the bitwidth of the parameters.<n>We first conduct a comprehensive reliability evaluation of quantized LLMs consisting of three key components.<n>A reliability peak occurs for 4-bit quantized models, indicating that quantizing moderately sized models offers the best reliability-efficiency trade-off.
  arXiv  Detail & Related papers  (2026-07-12T17:39:09Z)
• An Empirical Study of OpenPangu Quantization on Ascend NPUs [43.29474342134475]
  OpenPangu models are attractive targets for private and domestic large-language-model deployment.<n>This paper conducts a controlled empirical study of OpenPangu 1B and 7B models on Huawei Ascend 910B1 NPUs.
  arXiv  Detail & Related papers  (2026-06-19T09:33:11Z)
• Automated Proving of Shannon-Type Entropy Inequalities via Fine-Tuned Language Models and Guided Tree Search [50.16356451328644]
  Proving Shannon-type entropy inequalities is a fundamental task in information theory.<n>We investigate whether small-scale large language models, fine-tuned on atomic proof steps, can automate this process.<n>We show that GPT-5.5 solves 1.7% samples under zero-shot prompting while Psitip solves 33.3% samples.
  arXiv  Detail & Related papers  (2026-06-04T05:43:12Z)
• Measuring Five-Nines Reliability: Sample-Efficient LLM Evaluation in Saturated Benchmarks [45.86413490112477]
  Large language models (LLMs) are used in reliability-sensitive applications.<n>Estimateing rare failure probability with tight confidence bounds requires prohibitively large LLM inference sizes.<n>We propose to learn a sampling distribution concentrated on failure-prone inputs via the cross-entropy method.
  arXiv  Detail & Related papers  (2026-05-11T20:23:44Z)
• Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels [0.0]
  Large Language Models are routinely compressed via post-training quantization to reduce inference costs and memory footprint for cloud and edge deployment.<n>Existing studies typically compare only two conditions, rely on aggregate bias metrics, and evaluate a single model family.<n>We conduct a controlled empirical study of three instruction-tuned models at five precision levels (BF16 through 3-bit) on 12,148 BBQ bias benchmark items.
  arXiv  Detail & Related papers  (2026-05-02T05:41:47Z)
• Gemma 4, Phi-4, and Qwen3: Accuracy-Efficiency Tradeoffs in Dense and MoE Reasoning Language Models [6.396911723204044]
  Mixture-of-experts (MoE) language models are often expected to offer better quality-efficiency tradeoffs than dense models.<n>We present a benchmark of seven recent reasoning-oriented instruction-tuned models spanning dense and MoE designs.
  arXiv  Detail & Related papers  (2026-04-08T12:50:52Z)
• Fragile Reasoning: A Mechanistic Analysis of LLM Sensitivity to Meaning-Preserving Perturbations [2.7946918847372277]
  Large language models demonstrate strong performance on mathematical reasoning benchmarks, yet remain surprisingly fragile to meaning-preserving surface perturbations.<n>We systematically evaluate three open-weight LLMs, Mistral-7B, Llama-3-8B, and Qwen2.5-7B, on 677 GSM8K problems paired with semantically equivalent variants.<n>All three models exhibit substantial answer-flip rates (28.8%-45.1%), with number paraphrasing consistently more disruptive than name swaps.
  arXiv  Detail & Related papers  (2026-04-02T05:30:20Z)
• Edge Reliability Gap in Vision-Language Models: Quantifying Failure Modes of Compressed VLMs Under Visual Corruption [0.0]
  The rapid compression of large vision-language models for edge deployment raises an underexplored question: do compact models fail differently, not merely more often?<n>This study compares a 7-billion- parameter quantised VLM (Qwen2.5-VL-7B, 4-bit NF4) against a 500-million- parameter FP16 model (SmolVLM2-500M) across 4,000 samples from VQAv2 and COCO Captions.<n>A three-category error taxonomy (Object Blindness, Semantic Drift, Prior Bias) is applied as a diagnostic framework. Confidence calibration is measured via Expected Error (ECE) using mean token probability
  arXiv  Detail & Related papers  (2026-03-24T10:14:40Z)
• Compressed Causal Reasoning: Quantization and GraphRAG Effects on Interventional and Counterfactual Accuracy [0.0]
  This study systematically evaluate quantization effects across all three levels of Pearls Causal Ladder.<n>We find that rung level accuracy in Llama 3 8B remains broadly stable under quantization, with NF4 showing less than one percent overall degradation.<n>Experiments on the CRASS benchmark show near identical performance across precisions, indicating that existing commonsense counterfactual datasets lack the structural sensitivity needed to reveal quantization induced reasoning drift.
  arXiv  Detail & Related papers  (2025-12-13T17:54:15Z)
• Causal Understanding by LLMs: The Role of Uncertainty [43.87879175532034]
  Recent papers show LLMs achieve near-random accuracy in causal relation classification.<n>We investigate whether pretraining exposure to causal examples improves causal understanding.
  arXiv  Detail & Related papers  (2025-09-24T13:06:35Z)
• SEAL: Steerable Reasoning Calibration of Large Language Models for Free [58.931194824519935]
  Large Language Models (LLMs) have demonstrated compelling capabilities for complex reasoning tasks via the extended chain-of-thought (CoT) reasoning mechanism.<n>Recent studies reveal substantial redundancy in the CoT reasoning traces, which negatively impacts model performance.<n>We introduce SEAL, a training-free approach that seamlessly calibrates the CoT process, improving accuracy while demonstrating significant efficiency gains.
  arXiv  Detail & Related papers  (2025-04-07T02:42:07Z)
• ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantization [73.60493264901359]
  We present a unified framework for rigorous comparisons across 1-bit, 1.58-bit, 2-bit, 3-bit, and 4-bit quantization settings.<n>We show that ternary, 2-bit, and 3-bit quantization maintains comparable performance in the size-accuracy trade-off.<n>Considering hardware constraints, 2-bit quantization offers promising potential for memory reduction and speedup.
  arXiv  Detail & Related papers  (2025-02-04T18:59:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.