Quantifying Faithful Confidence Expression in Large Reasoning Models

• URL: http://arxiv.org/abs/2606.03969v1
• Date: Tue, 02 Jun 2026 17:53:45 GMT
• Title: Quantifying Faithful Confidence Expression in Large Reasoning Models
• Abstract summary: We introduce a novel framework to quantify faithful calibration of large reasoning models.<n>Our framework analyzes linguistic decisiveness relative to three sources of internal uncertainty.<n>We find that faithful confidence expression is a significant challenge for LRMs.
• Score: 40.863410553607984
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reliable uncertainty communication is critical to the trustworthiness of LLMs, yet faithful calibration (FC)--the alignment between models' intrinsic and (linguistically) expressed confidence--is a persistent failure mode. This challenge is key for large reasoning models (LRMs), whose extended reasoning traces are often interpreted by users as evidence of deliberation, competence, and confidence. Despite the importance of FC and wide usage of LRMs, the extent to which LRMs can faithfully express their confidence remains poorly understood. Moreover, the prevailing paradigm to measure FC does not generalize well to the long chain-of-thought outputs generated by LRMs, which tend to lack clear step boundaries, involve inconsistent step structure, and encode complex conditional dependencies throughout the trace--complicating estimation of intrinsic confidence. To address this challenge, we introduce a novel framework to systematically quantify FC of LRMs. Our framework analyzes linguistic decisiveness relative to three sources of internal uncertainty, based on token probabilities, hidden states, and sampled response consistency. We also devise a prefix-conditioned sampling approach to control for conditional and structural variation across traces. Applying our framework to a diverse suite of leading models, datasets, and prompts, we find that faithful confidence expression is a significant challenge for LRMs. Reasoning behaviors do not automatically translate to improved FC, and prompt interventions for non-reasoning models do not improve faithfulness in the reasoning setting. Different confidence estimators further produce divergent assessments of the same traces, revealing fragility in prior evaluation methodologies. Taken together, our work establishes FC as a distinct reliability and alignment target for LRMs, particularly as such systems are increasingly deployed in high-stakes contexts.

Related papers

• Can LLMs Use Linguistic Uncertainty Markers to Reliably Reflect Intrinsic Confidence? [50.17126900886782]
  It remains unclear whether models can apply their own linguistic confidence framework to associate markers with specific confidence levels in a stable and generalizable way.<n>We formalize _marker internal confidence_ (MIC) as the estimated intrinsic confidence a model associates with a specific epistemic marker in a given task domain.<n>Applying our analysis framework to diverse models and tasks, we find that LLMs remain faithfully miscalibrated even under model-centric interpretation of marker meanings.
  arXiv  Detail & Related papers  (2026-05-27T17:38:00Z)
• Process Supervision of Confidence Margin for Calibrated LLM Reasoning [52.373121066425455]
  Scaling test-time computation with reinforcement learning (RL) has emerged as a reliable path to improve large language models (LLM) reasoning ability.<n>Yet, outcome-based reward often incentivizes models to be overconfident, leading to hallucinations, unreliable confidence-based control, and unnecessary compute allocation.<n>We introduce Reinforcement Learning with Confidence Margin (textbfRLCM), a calibration-aware RL framework that jointly optimize correctness and confidence reliability.
  arXiv  Detail & Related papers  (2026-04-25T14:40:13Z)
• FACT-E: Causality-Inspired Evaluation for Trustworthy Chain-of-Thought Reasoning [49.65751420291115]
  Chain-of-Thought (CoT) prompting has improved LLM reasoning, but models often generate explanations that appear coherent while containing unfaithful intermediate steps.<n>We propose FACT-E, a causality-inspired framework for evaluating CoT quality.<n>We show that FACT-E improves reasoning-trajectory selection and yields stronger in-context learnings.
  arXiv  Detail & Related papers  (2026-04-12T15:35:08Z)
• BAS: A Decision-Theoretic Approach to Evaluating Large Language Model Confidence [22.382291859991472]
  We introduce the Behavioral Alignment Score (BAS), a decision-theoretic metric for evaluating how well large language models support abstention-aware decision making.<n>BAS is derived from an explicit answer-or-abstain utility model and aggregates realized utility across a continuum of risk thresholds.<n>We show theoretically that truthful confidence estimates uniquely maximize expected BAS utility, linking calibration to decision-optimal behavior.
  arXiv  Detail & Related papers  (2026-04-03T17:44:32Z)
• Fact-Checking with Large Language Models via Probabilistic Certainty and Consistency [7.806516365113592]
  Large language models (LLMs) are increasingly used in applications requiring factual accuracy.<n>While fact-checking can mitigate these errors, existing methods typically retrieve external evidence indiscriminately.<n>We introduce Probabilistic Certainty and Consistency (PCC), a framework that estimates factual confidence.
  arXiv  Detail & Related papers  (2026-01-05T21:57:41Z)
• FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoning [62.452350134196934]
  FaithCoT-Bench is a unified benchmark for instance-level CoT unfaithfulness detection.<n>Our framework formulates unfaithfulness detection as a discriminative decision problem.<n>FaithCoT-Bench sets a solid basis for future research toward more interpretable and trustworthy reasoning in LLMs.
  arXiv  Detail & Related papers  (2025-10-05T05:16:54Z)
• Calibrating Verbalized Confidence with Self-Generated Distractors [24.56911906044891]
  We introduce Distractor-Normalized Coherence (DINCO)<n>DINCO estimates and accounts for an LLM's suggestibility bias by having the model its confidence independently across several self-generated distractors.<n>We frame the popular approach of self-consistency as leveraging coherence across sampled generations, and normalized verbalized confidence as leveraging coherence across validations on incompatible claims.
  arXiv  Detail & Related papers  (2025-09-29T21:41:22Z)
• Temporalizing Confidence: Evaluation of Chain-of-Thought Reasoning with Signal Temporal Logic [0.12499537119440243]
  We propose a structured framework that models stepwise confidence as a temporal signal and evaluates it using Signal Temporal Logic (STL)<n>In particular, we define formal STL-based constraints to capture desirable temporal properties and compute scores that serve as structured, interpretable confidence estimates.<n>Our approach consistently improves calibration metrics and provides more reliable uncertainty estimates than conventional confidence aggregation and post-hoc calibration.
  arXiv  Detail & Related papers  (2025-06-09T21:21:12Z)
• ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation [91.20492150248106]
  We investigate the internal mechanisms behind unfaithful generation and identify a subset of mid-to-deep feed-forward networks (FFNs) that are disproportionately activated in such cases.<n>We propose Parametric Knowledge Muting through FFN Suppression (ParamMute), a framework that improves contextual faithfulness by suppressing the activation of unfaithfulness-associated FFNs.<n> Experimental results show that ParamMute significantly enhances faithfulness across both CoFaithfulQA and the established ConFiQA benchmark, achieving substantial reductions in reliance on parametric memory.
  arXiv  Detail & Related papers  (2025-02-21T15:50:41Z)
• Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Models [14.5291643644017]
  We introduce the concept of Confidence-Probability Alignment. We probe the alignment between models' internal and expressed confidence. Among the models analyzed, OpenAI's GPT-4 showed the strongest confidence-probability alignment.
  arXiv  Detail & Related papers  (2024-05-25T15:42:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.