Fact-Level Confidence Calibration and Self-Correction

• URL: http://arxiv.org/abs/2411.13343v1
• Date: Wed, 20 Nov 2024 14:15:18 GMT
• Title: Fact-Level Confidence Calibration and Self-Correction
• Authors: Yige Yuan, Bingbing Xu, Hexiang Tan, Fei Sun, Teng Xiao, Wei Li, Huawei Shen, Xueqi Cheng,
• Abstract summary: We propose a Fact-Level framework that calibrates confidence to relevance-weighted correctness at the fact level. We also develop Confidence-Guided Fact-level Self-Correction ($textbfConFix$), which uses high-confidence facts within a response as additional knowledge to improve low-confidence ones.
• Score: 64.40105513819272
• License:
• Abstract: Confidence calibration in LLMs, i.e., aligning their self-assessed confidence with the actual accuracy of their responses, enabling them to self-evaluate the correctness of their outputs. However, current calibration methods for LLMs typically estimate two scalars to represent overall response confidence and correctness, which is inadequate for long-form generation where the response includes multiple atomic facts and may be partially confident and correct. These methods also overlook the relevance of each fact to the query. To address these challenges, we propose a Fact-Level Calibration framework that operates at a finer granularity, calibrating confidence to relevance-weighted correctness at the fact level. Furthermore, comprehensive analysis under the framework inspired the development of Confidence-Guided Fact-level Self-Correction ($\textbf{ConFix}$), which uses high-confidence facts within a response as additional knowledge to improve low-confidence ones. Extensive experiments across four datasets and six models demonstrate that ConFix effectively mitigates hallucinations without requiring external knowledge sources such as retrieval systems.

Related papers

• Towards Fully Exploiting LLM Internal States to Enhance Knowledge Boundary Perception [58.62352010928591]
  Large language models (LLMs) exhibit impressive performance across diverse tasks but often struggle to accurately gauge their knowledge boundaries. This paper explores leveraging LLMs' internal states to enhance their perception of knowledge boundaries from efficiency and risk perspectives.
  arXiv  Detail & Related papers  (2025-02-17T11:11:09Z)
• Language Models Prefer What They Know: Relative Confidence Estimation via Confidence Preferences [62.52739672949452]
  Language models (LMs) should provide reliable confidence estimates to help users detect mistakes in their outputs and defer to human experts when necessary. We propose relative confidence estimation, where we match up questions against each other and ask the model to make relative judgments of confidence. Treating each question as a "player" in a series of matchups against other questions and the model's preferences as match outcomes, we can use rank aggregation methods like Elo rating and Bradley-Terry to translate the model's confidence preferences into confidence scores.
  arXiv  Detail & Related papers  (2025-02-03T07:43:27Z)
• Aligning Large Language Models for Faithful Integrity Against Opposing Argument [71.33552795870544]
  Large Language Models (LLMs) have demonstrated impressive capabilities in complex reasoning tasks. They can be easily misled by unfaithful arguments during conversations, even when their original statements are correct. We propose a novel framework, named Alignment for Faithful Integrity with Confidence Estimation.
  arXiv  Detail & Related papers  (2025-01-02T16:38:21Z)
• Confidence in the Reasoning of Large Language Models [0.0]
  Confidence is measured in terms of persistence in keeping their answer when prompted to reconsider. Confidence is only partially explained by the underlying token-level probability.
  arXiv  Detail & Related papers  (2024-12-19T10:04:29Z)
• Graph-based Confidence Calibration for Large Language Models [22.394717844099684]
  We propose a novel method to develop a well-calibrated confidence estimation model. We use a weighted graph to represent the consistency among the large language models' responses to a question. We then train a graph neural network to estimate the probability of correct responses.
  arXiv  Detail & Related papers  (2024-11-03T20:36:44Z)
• Enhancing Healthcare LLM Trust with Atypical Presentations Recalibration [20.049443396032423]
  Black-box large language models (LLMs) are increasingly deployed in various environments. LLMs often exhibit overconfidence, leading to potential risks and misjudgments. We propose a novel method, textitAtypical presentations Recalibration, which leverages atypical presentations to adjust the model's confidence estimates.
  arXiv  Detail & Related papers  (2024-09-05T03:45:35Z)
• SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationales [29.33581578047835]
  SaySelf is a training framework that teaches large language models to express more accurate fine-grained confidence estimates. In addition, SaySelf directs LLMs to produce self-reflective rationales that clearly identify gaps in their parametric knowledge. We show that the generated self-reflective rationales are reasonable and can further contribute to the calibration.
  arXiv  Detail & Related papers  (2024-05-31T16:21:16Z)
• When to Trust LLMs: Aligning Confidence with Response Quality [49.371218210305656]
  We propose CONfidence-Quality-ORDer-preserving alignment approach (CONQORD) It integrates quality reward and order-preserving alignment reward functions. Experiments demonstrate that CONQORD significantly improves the alignment performance between confidence and response accuracy.
  arXiv  Detail & Related papers  (2024-04-26T09:42:46Z)
• Fact-and-Reflection (FaR) Improves Confidence Calibration of Large Language Models [84.94220787791389]
  We propose Fact-and-Reflection (FaR) prompting, which improves the LLM calibration in two steps. Experiments show that FaR achieves significantly better calibration; it lowers the Expected Error by 23.5%. FaR even elicits the capability of verbally expressing concerns in less confident scenarios.
  arXiv  Detail & Related papers  (2024-02-27T01:37:23Z)
• Calibrating Long-form Generations from Large Language Models [34.72041258464477]
  Large Language Models' (LLMs) confidence scores should align with the actual likelihood of its responses being correct. Current confidence elicitation methods and calibration metrics rely on a binary true/false assessment of response correctness. We introduce a unified calibration framework, in which both the correctness of the LLMs' responses and their associated confidence levels are treated as distributions across a range of scores.
  arXiv  Detail & Related papers  (2024-02-09T17:00:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.