Bridging the Confidence Gap: Temperature Scaling for Calibrating Test-Time Prompt Tuning

• URL: http://arxiv.org/abs/2609.17386v1
• Date: Tue, 15 Sep 2026 16:25:48 GMT
• Title: Bridging the Confidence Gap: Temperature Scaling for Calibrating Test-Time Prompt Tuning
• Abstract summary: Test-time prompt tuning (TPT) enables adaptation on a single test instance, achieving improved accuracy but often sacrificing calibration performance.<n>Most existing calibration methods introduce additional regularization terms to promote dispersion across text embeddings.<n>We propose CoTS, a simple yet effective post-hoc calibration method that preserves accuracy.
• Score: 33.52650319991838
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Test-time prompt tuning (TPT) enables adaptation on a single test instance, achieving improved accuracy but often sacrificing calibration performance. Most existing calibration methods introduce additional regularization terms to promote dispersion across text embeddings and reduce calibration error, yet these methods often suffer from a drop in accuracy. Motivated by the well-calibrated nature of zero-shot predictions, we propose CoTS, a simple yet effective post-hoc calibration method that preserves accuracy. Specifically, CoTS applies temperature scaling to minimize the confidence gap between adapted and zero-shot predictions. To fully exploit the potential of multiple augmentations during adaptation, we introduce a weak-strong ensemble strategy that further boosts accuracy. We then apply CoTS to this ensemble, termed E-CoTS, to maintain its well-calibrated property. Extensive experiments on diverse datasets and backbones show that our approaches effectively mitigate miscalibration without compromising primary accuracy. For instance, E-CoTS reduces the average expected calibration error of TPT from 11.90% to 5.38% on ImageNet variants, while even increasing accuracy from 60.74% to 62.95%. Moreover, when integrated with existing calibration methods, E-CoTS usually enhances both accuracy and calibration simultaneously.

Related papers

• Respect Your Zero-Shot Uncertainty: Conservative Calibration for Test-Time-Adapted Vision-Language Models [5.63022899921334]
  We show that TTA can increase confidence and reduce entropy even when the top-1 prediction and its correctness remain unchanged.<n>We propose Zero-Shot-Anchored Entropy (ZAEC), a label-free post-hoc method that uses zero-shot entropy as a sample-specific uncertainty reference.
  arXiv  Detail & Related papers  (2026-08-06T12:11:53Z)
• Too Sharp, Too Sure: When Calibration Follows Curvature [39.687359662779826]
  Empirically, Expected Error (ECE) closely tracks curvature-based sharpness throughout optimization.<n>We show that both ECE and Gauss-Newton curvature are controlled, up to problem-specific constants.<n>We introduce a margin-aware training objective that explicitly targets robust-margin tails and local smoothness.
  arXiv  Detail & Related papers  (2026-04-22T14:28:52Z)
• Rethinking Early Stopping: Refine, Then Calibrate [49.966899634962374]
  We present a novel variational formulation of the calibration-refinement decomposition.<n>We provide theoretical and empirical evidence that calibration and refinement errors are not minimized simultaneously during training.
  arXiv  Detail & Related papers  (2025-01-31T15:03:54Z)
• Feature Clipping for Uncertainty Calibration [24.465567005078135]
  Modern deep neural networks (DNNs) often suffer from overconfidence, leading to miscalibration. We propose a novel post-hoc calibration method called feature clipping (FC) to address this issue. FC involves clipping feature values to a specified threshold, effectively increasing entropy in high calibration error samples.
  arXiv  Detail & Related papers  (2024-10-16T06:44:35Z)
• A Confidence Interval for the $\ell_2$ Expected Calibration Error [35.88784957918326]
  We develop confidence intervals $ell$ Expected the Error (ECE)<n>We consider top-1-to-$k$ calibration, which includes both the popular notion of confidence calibration as well as calibration.<n>For a debiased estimator of the ECE, we show normality, but with different convergence rates and variances for calibrated and misd models.
  arXiv  Detail & Related papers  (2024-08-16T20:00:08Z)
• C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature Dispersion [54.81141583427542]
  In deep learning, test-time adaptation has gained attention as a method for model fine-tuning without the need for labeled data. This paper explores calibration during test-time prompt tuning by leveraging the inherent properties of CLIP. We present a novel method, Calibrated Test-time Prompt Tuning (C-TPT), for optimizing prompts during test-time with enhanced calibration.
  arXiv  Detail & Related papers  (2024-03-21T04:08:29Z)
• Calibration by Distribution Matching: Trainable Kernel Calibration Metrics [56.629245030893685]
  We introduce kernel-based calibration metrics that unify and generalize popular forms of calibration for both classification and regression. These metrics admit differentiable sample estimates, making it easy to incorporate a calibration objective into empirical risk minimization. We provide intuitive mechanisms to tailor calibration metrics to a decision task, and enforce accurate loss estimation and no regret decisions.
  arXiv  Detail & Related papers  (2023-10-31T06:19:40Z)
• Sample-dependent Adaptive Temperature Scaling for Improved Calibration [95.7477042886242]
  Post-hoc approach to compensate for neural networks being wrong is to perform temperature scaling. We propose to predict a different temperature value for each input, allowing us to adjust the mismatch between confidence and accuracy. We test our method on the ResNet50 and WideResNet28-10 architectures using the CIFAR10/100 and Tiny-ImageNet datasets.
  arXiv  Detail & Related papers  (2022-07-13T14:13:49Z)
• Soft Calibration Objectives for Neural Networks [40.03050811956859]
  We propose differentiable losses to improve calibration based on a soft (continuous) version of the binning operation underlying popular calibration-error estimators. When incorporated into training, these soft calibration losses achieve state-of-the-art single-model ECE across multiple datasets with less than 1% decrease in accuracy.
  arXiv  Detail & Related papers  (2021-07-30T23:30:20Z)
• Parameterized Temperature Scaling for Boosting the Expressive Power in Post-Hoc Uncertainty Calibration [57.568461777747515]
  We introduce a novel calibration method, Parametrized Temperature Scaling (PTS) We demonstrate that the performance of accuracy-preserving state-of-the-art post-hoc calibrators is limited by their intrinsic expressive power. We show with extensive experiments that our novel accuracy-preserving approach consistently outperforms existing algorithms across a large number of model architectures, datasets and metrics.
  arXiv  Detail & Related papers  (2021-02-24T10:18:30Z)
• Localized Calibration: Metrics and Recalibration [133.07044916594361]
  We propose a fine-grained calibration metric that spans the gap between fully global and fully individualized calibration. We then introduce a localized recalibration method, LoRe, that improves the LCE better than existing recalibration methods.
  arXiv  Detail & Related papers  (2021-02-22T07:22:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.