Closing the Confidence-Faithfulness Gap in Large Language Models

• URL: http://arxiv.org/abs/2603.25052v2
• Date: Wed, 01 Apr 2026 05:05:49 GMT
• Title: Closing the Confidence-Faithfulness Gap in Large Language Models
• Abstract summary: Large language models verbalize confidence scores that are largely detached from their actual accuracy.<n>We present a mechanistic interpretability analysis of verbalized confidence.<n>We introduce a two-stage adaptive steering pipeline that reads the model's internal accuracy estimate and steers verbalized output to match it.
• Score: 3.122242683664973
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) tend to verbalize confidence scores that are largely detached from their actual accuracy, yet the geometric relationship governing this behavior remain poorly understood. In this work, we present a mechanistic interpretability analysis of verbalized confidence, using linear probes and contrastive activation addition (CAA) steering to show that calibration and verbalized confidence signals are encoded linearly but are orthogonal to one another -- a finding consistent across three open-weight models and four datasets. Interestingly, when models are prompted to simultaneously reason through a problem and verbalize a confidence score, the reasoning process disrupts the verbalized confidence direction, exacerbating miscalibration. We term this the "Reasoning Contamination Effect." Leveraging this insight, we introduce a two-stage adaptive steering pipeline that reads the model's internal accuracy estimate and steers verbalized output to match it, substantially improving calibration alignment across all evaluated models.

Related papers

• The Knowing-Saying Gap: When Probes See Errors that Confidence Misses [20.46211358172975]
  Linear probes detect corrupted context in language models with near-perfect accuracy, yet this does not translate into reliable failure prediction.<n>Across multi-hop arithmetic chains, probes that detect corruption turn out to be uninformative about final answer correctness.<n>As a real-time monitor, probe-based interventions are sharply model and error-type dependent.
  arXiv  Detail & Related papers  (2026-07-21T12:13:10Z)
• CaliDist: Calibrating Large Language Models via Behavioral Robustness to Distraction [51.56484100374058]
  Existing calibration methods for Large Language Models (LLMs) often overlook a critical dimension of trustworthiness: a model's behavioral robustness.<n>We introduce textscCaliDist, a novel post-hoc calibration approach that directly measures and penalizes a model's susceptibility to distraction.<n>textscCaliDist consistently achieves lower Expected Error (ECE) and Brier Score compared with strong baselines.
  arXiv  Detail & Related papers  (2026-06-04T07:27:53Z)
• Identifying Influential N-grams in Confidence Calibration via Regression Analysis [53.95767806659635]
  We identify what linguistic expressions are related to confidence by applying the regression method.<n>We show that large language models (LLMs) remain overconfident when reasoning is involved.<n>We reveal that confidence calibration is possible by simply suppressing those overconfident expressions without drops in performance.
  arXiv  Detail & Related papers  (2026-04-07T11:58:49Z)
• Decoding the Critique Mechanism in Large Reasoning Models [50.821607345799386]
  Large Reasoning Models (LRMs) exhibit backtracking and self-verification mechanisms that enable them to revise intermediate steps and reach correct solutions.<n>We investigate how current LRMs recover from errors by inserting arithmetic mistakes in their intermediate reasoning steps.<n>We find that despite the error propagating through the chain-of-thought, the model still reaches the correct final answer.
  arXiv  Detail & Related papers  (2026-03-17T10:03:30Z)
• On Calibration of Large Language Models: From Response To Capability [66.59139960234326]
  Large language models (LLMs) are widely deployed as general-purpose problem solvers.<n>We introduce capability calibration, which targets the model's expected accuracy on a query.<n>Our results demonstrate that capability-calibrated confidence improves pass@$k$ prediction and inference budget allocation.
  arXiv  Detail & Related papers  (2026-02-14T01:07:45Z)
• Direct Confidence Alignment: Aligning Verbalized Confidence with Internal Confidence In Large Language Models [6.918665116014629]
  Internal confidence of a model, derived from token probabilities, is not well aligned with its verbalized confidence.<n>We propose Direct Confidence Alignment (DCA) to align an LLM's verbalized confidence with its internal confidence.
  arXiv  Detail & Related papers  (2025-12-12T19:29:05Z)
• Verbalized Confidence Triggers Self-Verification: Emergent Behavior Without Explicit Reasoning Supervision [12.287123198288079]
  Uncertainty calibration is essential for the safe deployment of large language models (LLMs)<n>We find that supervised fine-tuning with scalar confidence labels alone suffices to elicit self-verification behavior of language models.<n>We propose a simple rethinking method that boosts performance via test-time scaling based on calibrated uncertainty.
  arXiv  Detail & Related papers  (2025-06-04T08:56:24Z)
• Reasoning Models Better Express Their Confidence [43.29929311958733]
  Large language models (LLMs) often fail to communicate their confidence accurately.<n>We demonstrate that reasoning models that engage in extended chain-of-thought (CoT) reasoning exhibit superior performance.
  arXiv  Detail & Related papers  (2025-05-20T15:19:00Z)
• Object-Level Verbalized Confidence Calibration in Vision-Language Models via Semantic Perturbation [26.580361841501514]
  Vision-language models (VLMs) excel in various multimodal tasks but frequently suffer from poor calibration.<n>This miscalibration undermines user trust, especially when models confidently provide incorrect or fabricated information.<n>We propose a novel Confidence through Semantic Perturbation (CSP) framework to improve the calibration of verbalized confidence for object-centric queries.
  arXiv  Detail & Related papers  (2025-04-21T04:01:22Z)
• Epistemic Integrity in Large Language Models [10.50127599111102]
  Large language models are increasingly relied upon sources of information, but their propensity for false or misleading statements poses high risks for users and society.<n>In this paper, we confront the critical problem of miscalibration where a model's linguistic assertiveness fails to reflect its true internal certainty.<n>We introduce a new human misalignment evaluation and a novel method for measuring the linguistic assertiveness of Large Language Models.
  arXiv  Detail & Related papers  (2024-11-10T17:10:13Z)
• Calibrating the Confidence of Large Language Models by Eliciting Fidelity [52.47397325111864]
  Large language models optimized with techniques like RLHF have achieved good alignment in being helpful and harmless. Post-alignment, these language models often exhibit overconfidence, where the expressed confidence does not accurately calibrate with their correctness rate. We propose a plug-and-play method to estimate the confidence of language models.
  arXiv  Detail & Related papers  (2024-04-03T11:36:12Z)
• Calibrating Large Language Models with Sample Consistency [76.23956851098598]
  We explore the potential of deriving confidence from the distribution of multiple randomly sampled model generations, via three measures of consistency. Results show that consistency-based calibration methods outperform existing post-hoc approaches. We offer practical guidance on choosing suitable consistency metrics for calibration, tailored to the characteristics of various LMs.
  arXiv  Detail & Related papers  (2024-02-21T16:15:20Z)
• Selective Learning: Towards Robust Calibration with Dynamic Regularization [79.92633587914659]
  Miscalibration in deep learning refers to there is a discrepancy between the predicted confidence and performance. We introduce Dynamic Regularization (DReg) which aims to learn what should be learned during training thereby circumventing the confidence adjusting trade-off.
  arXiv  Detail & Related papers  (2024-02-13T11:25:20Z)
• Robustness and Accuracy Could Be Reconcilable by (Proper) Definition [109.62614226793833]
  The trade-off between robustness and accuracy has been widely studied in the adversarial literature. We find that it may stem from the improperly defined robust error, which imposes an inductive bias of local invariance. By definition, SCORE facilitates the reconciliation between robustness and accuracy, while still handling the worst-case uncertainty.
  arXiv  Detail & Related papers  (2022-02-21T10:36:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.