Methods to Estimate Large Language Model Confidence

• URL: http://arxiv.org/abs/2312.03733v2
• Date: Fri, 8 Dec 2023 07:04:52 GMT
• Title: Methods to Estimate Large Language Model Confidence
• Authors: Maia Kotelanski, Robert Gallo, Ashwin Nayak, Thomas Savage
• Abstract summary: This study evaluates methods to measure Large Language Models confidence when suggesting a diagnosis for challenging clinical vignettes. SC Agreement Frequency is the most useful proxy for model confidence, especially for medical diagnosis.
• Score: 2.4797200957733576
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models have difficulty communicating uncertainty, which is a significant obstacle to applying LLMs to complex medical tasks. This study evaluates methods to measure LLM confidence when suggesting a diagnosis for challenging clinical vignettes. GPT4 was asked a series of challenging case questions using Chain of Thought and Self Consistency prompting. Multiple methods were investigated to assess model confidence and evaluated on their ability to predict the models observed accuracy. The methods evaluated were Intrinsic Confidence, SC Agreement Frequency and CoT Response Length. SC Agreement Frequency correlated with observed accuracy, yielding a higher Area under the Receiver Operating Characteristic Curve compared to Intrinsic Confidence and CoT Length analysis. SC agreement is the most useful proxy for model confidence, especially for medical diagnosis. Model Intrinsic Confidence and CoT Response Length exhibit a weaker ability to differentiate between correct and incorrect answers, preventing them from being reliable and interpretable markers for model confidence. We conclude GPT4 has a limited ability to assess its own diagnostic accuracy. SC Agreement Frequency is the most useful method to measure GPT4 confidence.

Related papers

• Enhancing Answer Reliability Through Inter-Model Consensus of Large Language Models [1.6874375111244329]
  We explore the collaborative dynamics of an innovative language model interaction system involving advanced models. These models generate and answer complex, PhD-level statistical questions without exact ground-truth answers. Our study investigates how inter-model consensus enhances the reliability and precision of responses.
  arXiv  Detail & Related papers  (2024-11-25T10:18:17Z)
• Fact-Level Confidence Calibration and Self-Correction [64.40105513819272]
  We propose a Fact-Level framework that calibrates confidence to relevance-weighted correctness at the fact level. We also develop Confidence-Guided Fact-level Self-Correction ($textbfConFix$), which uses high-confidence facts within a response as additional knowledge to improve low-confidence ones.
  arXiv  Detail & Related papers  (2024-11-20T14:15:18Z)
• Graph-based Confidence Calibration for Large Language Models [22.394717844099684]
  We propose a novel method to develop a well-calibrated confidence estimation model. We use a weighted graph to represent the consistency among the large language models' responses to a question. We then train a graph neural network to estimate the probability of correct responses.
  arXiv  Detail & Related papers  (2024-11-03T20:36:44Z)
• Uncertainty Estimation of Large Language Models in Medical Question Answering [60.72223137560633]
  Large Language Models (LLMs) show promise for natural language generation in healthcare, but risk hallucinating factually incorrect information. We benchmark popular uncertainty estimation (UE) methods with different model sizes on medical question-answering datasets. Our results show that current approaches generally perform poorly in this domain, highlighting the challenge of UE for medical applications.
  arXiv  Detail & Related papers  (2024-07-11T16:51:33Z)
• Interpretability of Uncertainty: Exploring Cortical Lesion Segmentation in Multiple Sclerosis [33.91263917157504]
  Uncertainty quantification (UQ) has become critical for evaluating the reliability of artificial intelligence systems. This study addresses the interpretability of instance-wise uncertainty values in deep learning models for focal lesion segmentation in magnetic resonance imaging.
  arXiv  Detail & Related papers  (2024-07-08T09:13:30Z)
• Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Models [14.5291643644017]
  We introduce the concept of Confidence-Probability Alignment. We probe the alignment between models' internal and expressed confidence. Among the models analyzed, OpenAI's GPT-4 showed the strongest confidence-probability alignment.
  arXiv  Detail & Related papers  (2024-05-25T15:42:04Z)
• Revisiting Confidence Estimation: Towards Reliable Failure Prediction [53.79160907725975]
  We find a general, widely existing but actually-neglected phenomenon that most confidence estimation methods are harmful for detecting misclassification errors. We propose to enlarge the confidence gap by finding flat minima, which yields state-of-the-art failure prediction performance.
  arXiv  Detail & Related papers  (2024-03-05T11:44:14Z)
• Word-Sequence Entropy: Towards Uncertainty Estimation in Free-Form Medical Question Answering Applications and Beyond [52.246494389096654]
  This paper introduces Word-Sequence Entropy (WSE), a method that calibrates uncertainty at both the word and sequence levels. We compare WSE with six baseline methods on five free-form medical QA datasets, utilizing seven popular large language models (LLMs)
  arXiv  Detail & Related papers  (2024-02-22T03:46:08Z)
• Selective Learning: Towards Robust Calibration with Dynamic Regularization [79.92633587914659]
  Miscalibration in deep learning refers to there is a discrepancy between the predicted confidence and performance. We introduce Dynamic Regularization (DReg) which aims to learn what should be learned during training thereby circumventing the confidence adjusting trade-off.
  arXiv  Detail & Related papers  (2024-02-13T11:25:20Z)
• Inadequacy of common stochastic neural networks for reliable clinical decision support [0.4262974002462632]
  Widespread adoption of AI for medical decision making is still hindered due to ethical and safety-related concerns. Common deep learning approaches, however, have the tendency towards overconfidence under data shift. This study investigates their actual reliability in clinical applications.
  arXiv  Detail & Related papers  (2024-01-24T18:49:30Z)
• Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback [91.22679548111127]
  A trustworthy real-world prediction system should produce well-calibrated confidence scores. We show that verbalized confidences emitted as output tokens are typically better-calibrated than the model's conditional probabilities.
  arXiv  Detail & Related papers  (2023-05-24T10:12:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.