Calibrating LLM Confidence with Semantic Steering: A Multi-Prompt Aggregation Framework

• URL: http://arxiv.org/abs/2503.02863v1
• Date: Tue, 04 Mar 2025 18:40:49 GMT
• Title: Calibrating LLM Confidence with Semantic Steering: A Multi-Prompt Aggregation Framework
• Authors: Ziang Zhou, Tianyuan Jin, Jieming Shi, Qing Li,
• Abstract summary: Large Language Models (LLMs) often exhibit misaligned confidence scores, usually overestimating the reliability of their predictions.<n>We propose a novel framework containing three components: confidence steering, steered confidence aggregation and steered answers selection.<n>We evaluate our method on 7 benchmarks and it consistently outperforms the baselines in terms of calibration metrics in task of confidence calibration and failure detection.
• Score: 11.872504642312705
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models (LLMs) often exhibit misaligned confidence scores, usually overestimating the reliability of their predictions. While verbalized confidence in Large Language Models (LLMs) has gained attention, prior work remains divided on whether confidence scores can be systematically steered through prompting. Recent studies even argue that such prompt-induced confidence shifts are negligible, suggesting LLMs' confidence calibration is rigid to linguistic interventions. Contrary to these claims, we first rigorously confirm the existence of directional confidence shifts by probing three models (including GPT3.5, LLAMA3-70b, GPT4) across 7 benchmarks, demonstrating that explicit instructions can inflate or deflate confidence scores in a regulated manner. Based on this observation, we propose a novel framework containing three components: confidence steering, steered confidence aggregation and steered answers selection, named SteeringConf. Our method, SteeringConf, leverages a confidence manipulation mechanism to steer the confidence scores of LLMs in several desired directions, followed by a summarization module that aggregates the steered confidence scores to produce a final prediction. We evaluate our method on 7 benchmarks and it consistently outperforms the baselines in terms of calibration metrics in task of confidence calibration and failure detection.

Related papers

• Language Models Prefer What They Know: Relative Confidence Estimation via Confidence Preferences [62.52739672949452]
  Language models (LMs) should provide reliable confidence estimates to help users detect mistakes in their outputs and defer to human experts when necessary.<n>We propose relative confidence estimation, where we match up questions against each other and ask the model to make relative judgments of confidence.<n>Treating each question as a "player" in a series of matchups against other questions and the model's preferences as match outcomes, we can use rank aggregation methods like Elo rating and Bradley-Terry to translate the model's confidence preferences into confidence scores.
  arXiv  Detail & Related papers  (2025-02-03T07:43:27Z)
• On Verbalized Confidence Scores for LLMs [25.160810008907397]
  Uncertainty quantification for large language models (LLMs) can establish more human trust into their responses. This work focuses on asking the LLM itself to verbalize its uncertainty with a confidence score as part of its output tokens. We assess the reliability of verbalized confidence scores with respect to different datasets, models, and prompt methods.
  arXiv  Detail & Related papers  (2024-12-19T11:10:36Z)
• Confidence in the Reasoning of Large Language Models [0.0]
  Confidence is measured in terms of persistence in keeping their answer when prompted to reconsider. Confidence is only partially explained by the underlying token-level probability.
  arXiv  Detail & Related papers  (2024-12-19T10:04:29Z)
• Fact-Level Confidence Calibration and Self-Correction [64.40105513819272]
  We propose a Fact-Level framework that calibrates confidence to relevance-weighted correctness at the fact level. We also develop Confidence-Guided Fact-level Self-Correction ($textbfConFix$), which uses high-confidence facts within a response as additional knowledge to improve low-confidence ones.
  arXiv  Detail & Related papers  (2024-11-20T14:15:18Z)
• Cycles of Thought: Measuring LLM Confidence through Stable Explanations [53.15438489398938]
  Large language models (LLMs) can reach and even surpass human-level accuracy on a variety of benchmarks, but their overconfidence in incorrect responses is still a well-documented failure mode. We propose a framework for measuring an LLM's uncertainty with respect to the distribution of generated explanations for an answer.
  arXiv  Detail & Related papers  (2024-06-05T16:35:30Z)
• SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationales [29.33581578047835]
  SaySelf is a training framework that teaches large language models to express more accurate fine-grained confidence estimates. In addition, SaySelf directs LLMs to produce self-reflective rationales that clearly identify gaps in their parametric knowledge. We show that the generated self-reflective rationales are reasonable and can further contribute to the calibration.
  arXiv  Detail & Related papers  (2024-05-31T16:21:16Z)
• Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Models [14.5291643644017]
  We introduce the concept of Confidence-Probability Alignment. We probe the alignment between models' internal and expressed confidence. Among the models analyzed, OpenAI's GPT-4 showed the strongest confidence-probability alignment.
  arXiv  Detail & Related papers  (2024-05-25T15:42:04Z)
• When to Trust LLMs: Aligning Confidence with Response Quality [49.371218210305656]
  We propose CONfidence-Quality-ORDer-preserving alignment approach (CONQORD) It integrates quality reward and order-preserving alignment reward functions. Experiments demonstrate that CONQORD significantly improves the alignment performance between confidence and response accuracy.
  arXiv  Detail & Related papers  (2024-04-26T09:42:46Z)
• Think Twice Before Trusting: Self-Detection for Large Language Models through Comprehensive Answer Reflection [90.71323430635593]
  We propose a novel self-detection paradigm that considers the comprehensive answer space beyond LLM-generated answers. Building upon this paradigm, we introduce a two-step framework, which firstly instructs LLM to reflect and provide justifications for each candidate answer. This framework can be seamlessly integrated with existing approaches for superior self-detection.
  arXiv  Detail & Related papers  (2024-03-15T02:38:26Z)
• Reconfidencing LLMs from the Grouping Loss Perspective [56.801251926946485]
  Large Language Models (LLMs) are susceptible to generating hallucinated answers in a confident tone. Recent findings show that controlling uncertainty must go beyond calibration. We construct a new evaluation dataset derived from a knowledge base to assess confidence scores given to answers of Mistral and LLaMA.
  arXiv  Detail & Related papers  (2024-02-07T15:40:22Z)
• Trust, but Verify: Using Self-Supervised Probing to Improve Trustworthiness [29.320691367586004]
  We introduce a new approach of self-supervised probing, which enables us to check and mitigate the overconfidence issue for a trained model. We provide a simple yet effective framework, which can be flexibly applied to existing trustworthiness-related methods in a plug-and-play manner.
  arXiv  Detail & Related papers  (2023-02-06T08:57:20Z)
• An evaluation of word-level confidence estimation for end-to-end automatic speech recognition [70.61280174637913]
  We investigate confidence estimation for end-to-end automatic speech recognition (ASR) We provide an extensive benchmark of popular confidence methods on four well-known speech datasets. Our results suggest a strong baseline can be obtained by scaling the logits by a learnt temperature.
  arXiv  Detail & Related papers  (2021-01-14T09:51:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.