Does Compression Preserve Uncertainty? A Unified Benchmark for Quantized and Sparse LLMs via Conformal Prediction

• URL: http://arxiv.org/abs/2606.01850v1
• Date: Mon, 01 Jun 2026 08:00:17 GMT
• Title: Does Compression Preserve Uncertainty? A Unified Benchmark for Quantized and Sparse LLMs via Conformal Prediction
• Abstract summary: We benchmark 12 large language models (LLMs) under various compression configurations across five NLP tasks.<n>Our experiments reveal that: (I) compression frequently decouples accuracy from uncertainty; (II) larger models absorb compression-induced uncertainty far more effectively than smaller ones; and (III) uncertainty inflation is often threshold-like rather than gradual.
• Score: 21.411353047208777
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Model compression techniques such as quantization and pruning are widely used to reduce the deployment cost of large language models (LLMs), with existing evaluations focusing almost exclusively on accuracy preservation. However, in safety-critical applications, a model's ability to reliably quantify its own uncertainty is equally important. We ask: does compression preserve this ability? To answer this question, we benchmark 12 LLMs under various compression configurations across five NLP tasks, using conformal prediction to provide a rigorous, distribution-free measure of uncertainty. Our experiments reveal that: (I) compression frequently decouples accuracy from uncertainty; (II) larger models absorb compression-induced uncertainty far more effectively than smaller ones; and (III) uncertainty inflation is often threshold-like rather than gradual. These results suggest that accuracy-only evaluation is insufficient for assessing the deployment readiness of compressed LLMs, and that uncertainty-aware benchmarking should be a standard component of model compression pipelines.

Related papers

• Benchmarking Composable Compression Techniques in Mixture-of-Experts LLMs [50.677556944160216]
  MoEXBench is a systematic benchmark for evaluating composable MoE compression as an end-to-end deployment workflow.<n>It measures composable-compression quality, workload and architecture, pruning/quantization/KV cache sensitivity, and deployment efficiency on commodity hardware.
  arXiv  Detail & Related papers  (2026-08-22T00:01:18Z)
• Just Noticeable Difference Modeling for Token Compression in Vision-Language-Action Models [62.522141464687195]
  We introduce Action-JND, which extends the principle of just noticeable difference (JND) to embodied perception.<n>A token change is considered tolerable only when the induced action deviation remains within a tolerated margin.<n>The resulting action-tolerance score serves as a plug-and-play criterion for VLA compression paradigms.
  arXiv  Detail & Related papers  (2026-08-21T15:59:37Z)
• The Asymmetric Harms of LLM Compression [9.45803162325743]
  Large language models (LLMs) compression reduces deployment costs, but standard aggregate metrics like perplexity and accuracy often mask underlying behavioral shifts.<n>We systematically evaluate 3 LLMs across 11 compression methods to investigate the effects of compression on knowledge retention, model confidence, and social bias.
  arXiv  Detail & Related papers  (2026-08-20T06:06:14Z)
• Statistically-Lossless Quantization of Large Language Models [41.38595517076645]
  This paper explores the middle ground of statistically-lossless compression through three complementary notions of losslessness for quantized LLMs.<n>First, task-lossless compression preserves zero-shot benchmark accuracy within natural sampling variance and remains achievable at aggressive bitwidths.<n>Second, we formalize the stricter notion of distribution-lossless compression, requiring the quantized model's next-token distribution to be practically indistinguishable from the original.<n>Third, we prove a gamma-squared variance law showing that symmetric quantization inflates noise variance by gamma squared relative to asymmetric quantization.
  arXiv  Detail & Related papers  (2026-05-04T09:46:47Z)
• Shorter, but Still Trustworthy? An Empirical Study of Chain-of-Thought Compression [19.669117846064562]
  Long chain-of-thought (Long-CoT) reasoning models have motivated a growing body of work on compressing reasoning traces to reduce inference cost.<n>We conduct the first systematic empirical study of how CoT compression affects model trustworthiness.<n>We find that CoT compression frequently introduces trustworthiness regressions and that different methods exhibit markedly different degradation profiles across dimensions.
  arXiv  Detail & Related papers  (2026-04-05T13:43:12Z)
• Arbitrary Ratio Feature Compression via Next Token Prediction [52.10426317889982]
  Arbitrary Ratio Feature Compression (ARFC) framework supports any compression ratio with a single model.<n>ARC is an auto-regressive model that performs compression via next-gressive prediction.<n>MoS module refines the compressed tokens by utilizing multiple compression results.<n>ERGC is integrated into the training process to preserve semantic and structural relationships during compression.
  arXiv  Detail & Related papers  (2026-02-12T02:38:57Z)
• Less Is More -- Until It Breaks: Security Pitfalls of Vision Token Compression in Large Vision-Language Models [69.84867664371826]
  We show that visual token compression substantially degrades the robustness of Large Vision-Language Models (LVLMs)<n>Small and imperceptible perturbations can significantly alter token importance ranking, leading the compression mechanism to mistakenly discard task-critical information.<n>We propose a Compression-Aware Attack to systematically study and exploit this vulnerability.
  arXiv  Detail & Related papers  (2026-01-17T13:02:41Z)
• Enhancing Trustworthiness with Mixed Precision: Benchmarks, Opportunities, and Challenges [12.438306093697]
  Large language models (LLMs) have shown promising performance across various tasks.<n>LLMs' autoregressive decoding process poses significant challenges for efficient deployment on existing AI hardware.
  arXiv  Detail & Related papers  (2025-11-27T14:17:43Z)
• Adaptive Individual Uncertainty under Out-Of-Distribution Shift with Expert-Routed Conformal Prediction [8.306260695214972]
  We introduce Trustworthy Expert Split-conformal with Scaled Estimation for Efficient Reliable Adaptive intervals (TESSERA)<n>TESSERA provides per-sample uncertainty with reliable coverage guarantee, informative and adaptive prediction interval widths that track the absolute error.<n>We evaluate on protein-ligand binding affinity prediction under both independent and identically distributed (i.i.d.) and scaffold-based out-of-distribution (OOD) splits.
  arXiv  Detail & Related papers  (2025-10-17T01:51:33Z)
• UNComp: Can Matrix Entropy Uncover Sparsity? -- A Compressor Design from an Uncertainty-Aware Perspective [85.08718140718707]
  UNComp is an uncertainty-aware framework that uncovers sparsity patterns that can be used for adaptive compression.<n>By focusing on uncertainty to analyze the sparsity pattern in detail, UNComp reduces the KV cache size to 4.74% of the original, achieves a 6% prefill speedup, and improves throughput by 6.4x.
  arXiv  Detail & Related papers  (2024-10-04T02:32:36Z)
• Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compression [109.23761449840222]
  This study conducts the first, thorough evaluation of leading Large Language Models (LLMs) We find that quantization is currently a more effective approach than pruning in achieving efficiency and trustworthiness simultaneously.
  arXiv  Detail & Related papers  (2024-03-18T01:38:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.