Quantile Adaptive Temperature Scaling for Confidence Calibration

• URL: http://arxiv.org/abs/2606.21749v1
• Date: Fri, 19 Jun 2026 21:06:11 GMT
• Title: Quantile Adaptive Temperature Scaling for Confidence Calibration
• Abstract summary: We introduce Quantile Adaptive Temperature Scaling (QaTS), a simple and efficient post hoc calibration method.<n>QaTS normalizes the calibration problem, makes the structure of miscalibration explicit and enables a monotone temperature function.<n>It delivers more reliable and trustworthy confidence estimates without modifying model predictions.
• Score: 32.74737063872657
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep neural networks often produce poorly calibrated confidence estimates, overstating their certainty even when predictions are incorrect. Temperature Scaling remains the most widely used posthoc calibration method due to its simplicity and effectiveness, yet its global, uniform rescaling of logits fails to correct the highly heterogeneous structure of miscalibration observed across the confidence spectrum. In particular, the largest correctness confidence discrepancies arise in different quantile regions depending on the setting, low confidence predictions, where uncertainty matters most, tend to exhibit the largest correctness confidence discrepancies, which standard TS leaves largely unaddressed. We introduce Quantile Adaptive Temperature Scaling (QaTS), a simple and efficient post hoc calibration method that adapts the temperature as a function of a predictions empirical confidence quantile. By mapping confidences into the quantile space, QaTS normalizes the calibration problem, makes the structure of miscalibration explicit and enables a monotone temperature function that adapts across quantiles while leaving well calibrated high confidence predictions largely unchanged. preserving high confidence behavior. This quantile aware formulation aligns naturally with a reparameterized Expected Calibration Error (ECE) objective and yields a sample wise temperature that is robust across a variety of challenging scenarios, such as class imbalance and distributional shifts. Across a broad range of datasets, architectures, evaluation scenarios and diverse tasks, QaTS consistently, and substantially, outperforms state of the art post hoc calibration methods, delivering more reliable and trustworthy confidence estimates without modifying model predictions.

Related papers

• Expectation Consistency Loss: Rethink Confidence Calibration under Covariate Shift [7.333172294727063]
  General confidence calibration methods assume training and test data are independent and identically distributed.<n>This paper proposes an unsupervised domain adaptation loss to calibrate confidence of the target domain.<n>We prove that computing ECL loss has the same sample complexity as Expected Error (ECE) and provide a theoretically grounded mini-batch trainable scheme for ECL loss.
  arXiv  Detail & Related papers  (2026-05-20T11:45:39Z)
• Improving Semantic Uncertainty Quantification in Language Model Question-Answering via Token-Level Temperature Scaling [55.29156356409814]
  We show that current approaches, particularly fixed-temperatures, produce systematically miscalibrated and poorly discriminative semantic confidence distributions.<n>We demonstrate that optimising a single temperature, which, we argue, provides a suitable inductive bias, is a surprisingly simple yet effective solution.
  arXiv  Detail & Related papers  (2026-04-08T15:00:32Z)
• On Calibration of Large Language Models: From Response To Capability [66.59139960234326]
  Large language models (LLMs) are widely deployed as general-purpose problem solvers.<n>We introduce capability calibration, which targets the model's expected accuracy on a query.<n>Our results demonstrate that capability-calibrated confidence improves pass@$k$ prediction and inference budget allocation.
  arXiv  Detail & Related papers  (2026-02-14T01:07:45Z)
• Temperature Scaling Attack Disrupting Model Confidence in Federated Learning [4.863985119779627]
  We present the Temperature Scaling Attack (TSA), a training-time attack that degrades calibration while preserving accuracy.<n>Our results establish calibration integrity as a critical attack surface in federated learning.
  arXiv  Detail & Related papers  (2026-02-06T12:01:54Z)
• Calibrating Language Models with Adaptive Temperature Scaling [58.056023173579625]
  We introduce Adaptive Temperature Scaling (ATS), a post-hoc calibration method that predicts a temperature scaling parameter for each token prediction. ATS improves calibration by over 10-50% across three downstream natural language evaluation benchmarks compared to prior calibration methods.
  arXiv  Detail & Related papers  (2024-09-29T22:54:31Z)
• A Confidence Interval for the $\ell_2$ Expected Calibration Error [35.88784957918326]
  We develop confidence intervals $ell$ Expected the Error (ECE)<n>We consider top-1-to-$k$ calibration, which includes both the popular notion of confidence calibration as well as calibration.<n>For a debiased estimator of the ECE, we show normality, but with different convergence rates and variances for calibrated and misd models.
  arXiv  Detail & Related papers  (2024-08-16T20:00:08Z)
• Does confidence calibration improve conformal prediction? [10.340903334800787]
  We show that current confidence calibration methods lead to larger prediction sets in adaptive conformal prediction.<n>By investigating the role of temperature value, we observe that high-confidence predictions can enhance the efficiency of adaptive conformal prediction.<n>We propose Conformal Temperature Scaling (ConfTS), a variant of temperature scaling with a novel loss function designed to enhance the efficiency of prediction sets.
  arXiv  Detail & Related papers  (2024-02-06T19:27:48Z)
• Calibration of Neural Networks [77.34726150561087]
  This paper presents a survey of confidence calibration problems in the context of neural networks. We analyze problem statement, calibration definitions, and different approaches to evaluation. Empirical experiments cover various datasets and models, comparing calibration methods according to different criteria.
  arXiv  Detail & Related papers  (2023-03-19T20:27:51Z)
• Sample-dependent Adaptive Temperature Scaling for Improved Calibration [95.7477042886242]
  Post-hoc approach to compensate for neural networks being wrong is to perform temperature scaling. We propose to predict a different temperature value for each input, allowing us to adjust the mismatch between confidence and accuracy. We test our method on the ResNet50 and WideResNet28-10 architectures using the CIFAR10/100 and Tiny-ImageNet datasets.
  arXiv  Detail & Related papers  (2022-07-13T14:13:49Z)
• Parameterized Temperature Scaling for Boosting the Expressive Power in Post-Hoc Uncertainty Calibration [57.568461777747515]
  We introduce a novel calibration method, Parametrized Temperature Scaling (PTS) We demonstrate that the performance of accuracy-preserving state-of-the-art post-hoc calibrators is limited by their intrinsic expressive power. We show with extensive experiments that our novel accuracy-preserving approach consistently outperforms existing algorithms across a large number of model architectures, datasets and metrics.
  arXiv  Detail & Related papers  (2021-02-24T10:18:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.