Reported Confidence in LLMs Tracks Commitment More Than Correctness

• URL: http://arxiv.org/abs/2606.29490v1
• Date: Sun, 28 Jun 2026 16:39:18 GMT
• Title: Reported Confidence in LLMs Tracks Commitment More Than Correctness
• Abstract summary: Confidence is an estimate of the probability that a chosen answer is correct.<n>We test this with a two-stage abstention paradigm from the neuroscience of perceptual decision making.<n> verbal confidence predicted the commit/abstain decision substantially better than whether the answer was correct.
• Score: 0.7783842564949857
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Confidence is an estimate of the probability that a chosen answer is correct. Verbal confidence reports are widely used as uncertainty measures in large language models, but whether they are best understood as estimates of correctness is unclear. We test this with a two-stage abstention paradigm from the neuroscience of perceptual decision making: a model first answers and reports its confidence, then decides whether to commit it to a user or abstain. Across four non-reasoning models, prompt framings, and confidence formats, verbal confidence predicted the commit/abstain decision substantially better than whether the answer was correct. Calibrated token log-probabilities showed the opposite profile, with abstention-prediction coupled to correctness discrimination, the signature of an answer-evidence signal. After removing the variance verbal confidence shared with log-probabilities, the residual stayed aligned with commitment while its link to correctness fell to near chance. The dissociation generalised to four reasoning models across four benchmarks of varying difficulty, from hard multiple-choice to frontier-level freeform questions. Mechanistic analyses in Gemma 3 and 4 were convergent: a post-answer state known to causally support verbal-confidence generation already encoded the future abstention decision before the abstention prompt, organised mainly by that decision rather than by correctness, the two lying in approximately orthogonal directions in activation space. Steering along a verbal-confidence-specific direction causally shifted abstention. Verbal and log-probability confidence are thus not interchangeable: log-probabilities track answer evidence and correctness, whereas verbal confidence is better understood as a behaviour-facing readout of an internal commit-readiness state, challenging the practice of treating verbal reports as proxies for reliability.

Related papers

• Claim-Level Confidence Calibration for Reliable Decision Making with Large Language Models [7.714704554667688]
  Large Language Models (LLMs) increasingly support decision-making in high-stakes domains.<n>LLMs often hallucinate and express confidence that is misaligned with factual correctness.<n>We study claim-level confidence calibration as a decision-relevant uncertainty signal.
  arXiv  Detail & Related papers  (2026-08-23T16:09:54Z)
• DirEAG: Dirichlet Evidence Aggregation for Calibrating Verbalized Confidence in Mathematical Reasoning [2.867517731896504]
  Black-box verbalized confidence is difficult to calibrate in large language models.<n>We propose DirEAG, a Dirichlet Evidence Aggregation method that converts each elicited answer-confidence observation into calibrated soft evidence.
  arXiv  Detail & Related papers  (2026-08-21T03:48:25Z)
• The Computational Basis of Confidence in Large Language Models [2.8220151940446425]
  We use statistical decision confidence to study confidence in language models.<n>We show that answer logits behave as monotonic readouts of a latent decision variable.<n>These results establish SDC as a unifying framework for studying confidence across biological and artificial intelligence.
  arXiv  Detail & Related papers  (2026-07-14T07:24:32Z)
• Asking Is Not Enough: Protocol Sensitivity in LLM Confidence Calibration [3.450547277166974]
  LLM confidence calibration is often evaluated by comparing two signals: token-probability scores and verbalized confidence.<n>We vary the measurement axes that define the verbalized-vs-token comparison.<n>We argue that both confidence signals should be treated as protocol-dependent behavioral measurements.
  arXiv  Detail & Related papers  (2026-05-26T23:03:38Z)
• Understanding and Mitigating Premature Confidence for Better LLM Reasoning [76.16007941549857]
  Long chains of thought (CoT) from current language models frequently contain logical gaps and unjustified leaps.<n>We find such a signal in how the model's confidence evolves during reasoning.<n>We exploit this in progressive confidence shaping, a reinforcement learning objective that trains models to update their confidence as they reason rather than commit early.
  arXiv  Detail & Related papers  (2026-05-23T04:42:45Z)
• Retrieval-Augmented Linguistic Calibration [57.41519309308438]
  We model linguistic confidence as a distribution over plausible perceived probability values that a statement is correct.<n>We present Retrieval-Augmented Linguistic truth (RALC), a lightweight post-hoc pipeline that propagates confidence signals back into natural language.
  arXiv  Detail & Related papers  (2026-05-19T04:31:38Z)
• ORCE: Order-Aware Alignment of Verbalized Confidence in Large Language Models [17.761630515662947]
  We propose a decoupled and order-aware framework for verbalized confidence calibration.<n>Our method first generates an answer and then estimates confidence conditioned on the fixed question--answer pair.<n>Experiments show that our method improves calibration and failure prediction performance while largely preserving answer accuracy.
  arXiv  Detail & Related papers  (2026-05-12T17:39:43Z)
• Confidence Before Answering: A Paradigm Shift for Efficient LLM Uncertainty Estimation [47.91529693614168]
  Existing methods are predominantly answer-first, producing confidence only after generating an answer.<n>We study a confidence-first paradigm, where the model outputs its confidence before answering, interpreting this score as the model's probability of answering the question correctly.<n>We propose CoCA, a reinforcement learning framework that jointly optimize confidence calibration and answer accuracy via segmented credit assignment.
  arXiv  Detail & Related papers  (2026-03-06T04:03:13Z)
• Critical or Compliant? The Double-Edged Sword of Reasoning in Chain-of-Thought Explanations [60.27156500679296]
  We study the role of Chain-of-Thought (CoT) explanations in moral scenarios by systematically perturbing reasoning chains and manipulating delivery tones.<n>Our findings reveal two key effects: (1) users often trust with outcome agreement, sustaining reliance even when reasoning is flawed.<n>These results highlight how CoT explanations can simultaneously clarify and mislead, underscoring the need for NLP systems to provide explanations that encourage scrutiny and critical thinking rather than blind trust.
  arXiv  Detail & Related papers  (2025-11-15T02:38:49Z)
• ConfTuner: Training Large Language Models to Express Their Confidence Verbally [58.63318088243125]
  Large Language Models (LLMs) are increasingly deployed in high-stakes domains such as science, law, and healthcare.<n>LLMs are often observed to generate incorrect answers with high confidence, a phenomenon known as "overconfidence"
  arXiv  Detail & Related papers  (2025-08-26T09:25:32Z)
• Uncertainty Distillation: Teaching Language Models to Express Semantic Confidence [16.311538811237536]
  Large language models (LLMs) are increasingly used for factual question-answering.<n>For these verbalized expressions of uncertainty to be meaningful, they should reflect the error rates at the expressed level of confidence.<n>We propose a simple procedure, uncertainty distillation, to teach an LLM to calibrated semantic confidences.
  arXiv  Detail & Related papers  (2025-03-18T21:29:29Z)
• Language Models Prefer What They Know: Relative Confidence Estimation via Confidence Preferences [62.52739672949452]
  Language models (LMs) should provide reliable confidence estimates to help users detect mistakes in their outputs and defer to human experts when necessary.<n>We propose relative confidence estimation, where we match up questions against each other and ask the model to make relative judgments of confidence.<n>Treating each question as a "player" in a series of matchups against other questions and the model's preferences as match outcomes, we can use rank aggregation methods like Elo rating and Bradley-Terry to translate the model's confidence preferences into confidence scores.
  arXiv  Detail & Related papers  (2025-02-03T07:43:27Z)
• Think Twice Before Trusting: Self-Detection for Large Language Models through Comprehensive Answer Reflection [90.71323430635593]
  We propose a novel self-detection paradigm that considers the comprehensive answer space beyond LLM-generated answers. Building upon this paradigm, we introduce a two-step framework, which firstly instructs LLM to reflect and provide justifications for each candidate answer. This framework can be seamlessly integrated with existing approaches for superior self-detection.
  arXiv  Detail & Related papers  (2024-03-15T02:38:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.