ConfidenceBench: Evaluating Confidence Calibration in Large Language Models

• URL: http://arxiv.org/abs/2607.20526v1
• Date: Fri, 10 Jul 2026 01:56:30 GMT
• Title: ConfidenceBench: Evaluating Confidence Calibration in Large Language Models
• Abstract summary: ConfidenceBench is a calibration benchmark for large language models.<n>It evaluates verbalized confidence estimates in 15 frontier large language models.<n> Confidence is elicited via prompting, requiring no access to model logits.
• Score: 14.831793255747181
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) are increasingly deployed in settings where fluent but incorrect answers can be costly. In these settings, accuracy alone is insufficient: models must also know when they are likely to be wrong. We present ConfidenceBench, a calibration benchmark that evaluates verbalized confidence estimates in 15 frontier LLMs using the Brier score, a proper scoring rule that incentivises truthful probability reporting. Confidence is elicited via prompting, requiring no access to model logits and making the framework applicable to both closed-source and open-source systems. The benchmark comprises 200 private multiple-choice questions across four categories: spatial reasoning, high-precision mathematics, word lookup, and unknowable questions. Across three independent runs, Claude Opus 4.6 and Gemini 3.1 Pro Preview achieve the lowest Brier scores, both reported as 0.103. Both substantially outperform the calibrated-random baseline of 0.1875, while Gemini 3.1 Flash-Lite scores 0.367, indicating severe miscalibration. Accuracy and calibration diverge substantially across model families: the most accurate model is not the best-calibrated, and several models perform worse than the calibrated-random Brier baseline despite reasonable accuracy. These results show that verbalized confidence calibration is a distinct and practically important axis of LLM reliability, complementary to standard accuracy-based evaluation.

Related papers

• Provable Limits and Certified Deferral for Verbalized Uncertainty in Small Language Models [0.0]
  Small open-weight language models increasingly run in private, offline, and cost-sensitive settings.<n>We study whether verbalized confidence can support risk-controlled deferral.
  arXiv  Detail & Related papers  (2026-08-05T17:11:04Z)
• Can You Trust the Confidence? ConfBench for Vision-Language Models on Document Extraction [13.176724997257802]
  Existing document benchmarks are dominated by clean, high-quality samples.<n>We introduce ConfBench, the first calibration-specific benchmark for key information extraction.<n>We evaluate four proprietary and three open-weight VLMs under verbalized and log-probability confidence estimation methods.
  arXiv  Detail & Related papers  (2026-08-03T07:03:51Z)
• The ACUTE Protocol: Operationalizing Language Model Activations for Better Calibration, Utility, and Trust [39.08747366439304]
  We develop a new metric, expected utility renormalized by the oracle (EURO), that balances calibration and informativeness.<n>We also propose a general-purpose-based confidence, utility, and trust estimation protocol (ACUTE) to appropriately adjudicate uncertainty.
  arXiv  Detail & Related papers  (2026-06-05T20:15:50Z)
• CaliDist: Calibrating Large Language Models via Behavioral Robustness to Distraction [51.56484100374058]
  Existing calibration methods for Large Language Models (LLMs) often overlook a critical dimension of trustworthiness: a model's behavioral robustness.<n>We introduce textscCaliDist, a novel post-hoc calibration approach that directly measures and penalizes a model's susceptibility to distraction.<n>textscCaliDist consistently achieves lower Expected Error (ECE) and Brier Score compared with strong baselines.
  arXiv  Detail & Related papers  (2026-06-04T07:27:53Z)
• Calibration vs Decision Making: Revisiting the Reliability Paradox in Unlearned Language Models [15.63775421273161]
  We investigate the gap in generative language models using the multiple-choice question-answering evaluation protocol on the TOFU benchmark.<n>We find that fine-tuned models achieve low calibration error (ECE 0.04) compared to pretrained models.<n>Models after unlearning retain similarly low calibration despite reduced accuracy on the forget split.
  arXiv  Detail & Related papers  (2026-05-20T08:59:23Z)
• Discovery of Hidden Miscalibration Regimes [52.452902154360565]
  Models may be systematically overconfident on some kinds of inputs and underconfident on others.<n>We define the corresponding miscalibration field and propose a diagnostic framework for estimating it.<n>Our approach learns a calibration-aware representation of the input space and estimates signed local miscalibration by kernel smoothing in the learned geometry.
  arXiv  Detail & Related papers  (2026-05-13T13:07:50Z)
• What Single-Prompt Accuracy Misses: A Multi-Variant Reliability Audit of Language Models [0.0]
  Single-prompt accuracy is the dominant way to benchmark language models, but it can miss reliability failures that matter.<n>We evaluate a 15-model open-weight corpus, with the main reliability analyses focused on 10 instruct models across five classification and reasoning benchmarks.<n>We find three broad results. First, evaluation design can materially change the conclusion.<n>Second, confidence signals are fragile. On MMLU-Pro, every primary model verbally reports confidence substantially above both its accuracy and its token-probability confidence on the same rows, and verbal parse rate can collapse for a single model on a single prompt variant.
  arXiv  Detail & Related papers  (2026-05-03T20:05:08Z)
• When Can We Trust LLM Graders? Calibrating Confidence for Automated Assessment [1.896929753246251]
  Large Language Models (LLMs) show promise for automated grading, but their outputs can be unreliable.<n>This enables selective automation where high-confidence predictions are processed automatically while uncertain cases are flagged for human review.
  arXiv  Detail & Related papers  (2026-03-31T10:38:22Z)
• On Calibration of Large Language Models: From Response To Capability [66.59139960234326]
  Large language models (LLMs) are widely deployed as general-purpose problem solvers.<n>We introduce capability calibration, which targets the model's expected accuracy on a query.<n>Our results demonstrate that capability-calibrated confidence improves pass@$k$ prediction and inference budget allocation.
  arXiv  Detail & Related papers  (2026-02-14T01:07:45Z)
• Do Large Language Models Know What They Don't Know? Kalshibench: A New Benchmark for Evaluating Epistemic Calibration via Prediction Markets [0.0]
  A well-calibrated model should express confidence that matches its actual accuracy -- when it claims 80% confidence, it should be correct 80% of the time.<n>We introduce textbfKalshiBench, a benchmark of 300 prediction market questions from Kalshi, a CFTC-regulated exchange.<n>We evaluate five frontier models -- Claude Opus 4.5, GPT-5.2, DeepSeek-V3.2, Qwen3-235B, and Kimi-K2 -- and find textbfsystematic overconfidence across all models.
  arXiv  Detail & Related papers  (2025-12-17T23:23:06Z)
• Annotation-Efficient Universal Honesty Alignment [70.05453324928955]
  Existing methods either rely on training-free confidence estimation or training-based calibration with correctness annotations.<n>Elicitation-Then-Calibration (EliCal) is a two-stage framework that first elicits internal confidence using inexpensive self-consistency supervision, then calibrates this confidence with a small set of correctness annotations.<n>EliCal achieves near-optimal alignment with only 1k correctness annotations (0.18% of full supervision) and better alignment performance on unseen MMLU tasks than the calibration-only baseline.
  arXiv  Detail & Related papers  (2025-10-20T13:05:22Z)
• ConfTuner: Training Large Language Models to Express Their Confidence Verbally [58.63318088243125]
  Large Language Models (LLMs) are increasingly deployed in high-stakes domains such as science, law, and healthcare.<n>LLMs are often observed to generate incorrect answers with high confidence, a phenomenon known as "overconfidence"
  arXiv  Detail & Related papers  (2025-08-26T09:25:32Z)
• Calibrating Large Language Models with Sample Consistency [76.23956851098598]
  We explore the potential of deriving confidence from the distribution of multiple randomly sampled model generations, via three measures of consistency. Results show that consistency-based calibration methods outperform existing post-hoc approaches. We offer practical guidance on choosing suitable consistency metrics for calibration, tailored to the characteristics of various LMs.
  arXiv  Detail & Related papers  (2024-02-21T16:15:20Z)
• Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback [91.22679548111127]
  A trustworthy real-world prediction system should produce well-calibrated confidence scores. We show that verbalized confidences emitted as output tokens are typically better-calibrated than the model's conditional probabilities.
  arXiv  Detail & Related papers  (2023-05-24T10:12:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.