To Believe or Not to Believe Your LLM

• URL: http://arxiv.org/abs/2406.02543v2
• Date: Wed, 17 Jul 2024 15:55:51 GMT
• Title: To Believe or Not to Believe Your LLM
• Authors: Yasin Abbasi Yadkori, Ilja Kuzborskij, András György, Csaba Szepesvári,
• Abstract summary: We explore uncertainty quantification in large language models (LLMs) We derive an information-theoretic metric that allows to reliably detect when only epistemic uncertainty is large. We conduct a series of experiments which demonstrate the advantage of our formulation.
• Score: 51.2579827761899
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We explore uncertainty quantification in large language models (LLMs), with the goal to identify when uncertainty in responses given a query is large. We simultaneously consider both epistemic and aleatoric uncertainties, where the former comes from the lack of knowledge about the ground truth (such as about facts or the language), and the latter comes from irreducible randomness (such as multiple possible answers). In particular, we derive an information-theoretic metric that allows to reliably detect when only epistemic uncertainty is large, in which case the output of the model is unreliable. This condition can be computed based solely on the output of the model obtained simply by some special iterative prompting based on the previous responses. Such quantification, for instance, allows to detect hallucinations (cases when epistemic uncertainty is high) in both single- and multi-answer responses. This is in contrast to many standard uncertainty quantification strategies (such as thresholding the log-likelihood of a response) where hallucinations in the multi-answer case cannot be detected. We conduct a series of experiments which demonstrate the advantage of our formulation. Further, our investigations shed some light on how the probabilities assigned to a given output by an LLM can be amplified by iterative prompting, which might be of independent interest.

Related papers

• On Subjective Uncertainty Quantification and Calibration in Natural Language Generation [2.622066970118316]
  Large language models often involve the generation of free-form responses, in which case uncertainty quantification becomes challenging. This work addresses these challenges from a perspective of Bayesian decision theory. We discuss how this assumption enables principled quantification of the model's subjective uncertainty and its calibration. The proposed methods can be applied to black-box language models.
  arXiv  Detail & Related papers  (2024-06-07T18:54:40Z)
• Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities [79.9629927171974]
  Uncertainty in Large Language Models (LLMs) is crucial for applications where safety and reliability are important. We propose Kernel Language Entropy (KLE), a novel method for uncertainty estimation in white- and black-box LLMs.
  arXiv  Detail & Related papers  (2024-05-30T12:42:05Z)
• Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantification [116.77055746066375]
  Large language models (LLMs) are notorious for hallucinating, i.e., producing erroneous claims in their output. We propose a novel fact-checking and hallucination detection pipeline based on token-level uncertainty quantification.
  arXiv  Detail & Related papers  (2024-03-07T17:44:17Z)
• Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling [69.83976050879318]
  In large language models (LLMs), identifying sources of uncertainty is an important step toward improving reliability, trustworthiness, and interpretability. In this paper, we introduce an uncertainty decomposition framework for LLMs, called input clarification ensembling. Our approach generates a set of clarifications for the input, feeds them into an LLM, and ensembles the corresponding predictions.
  arXiv  Detail & Related papers  (2023-11-15T05:58:35Z)
• Postselection-free learning of measurement-induced quantum dynamics [0.0]
  We introduce a general-purpose scheme that can be used to infer any property of the post-measurement ensemble of states. As an immediate application, we show that our method can be used to verify the emergence of quantum state designs in experiments.
  arXiv  Detail & Related papers  (2023-10-06T11:06:06Z)
• Exploiting Independent Instruments: Identification and Distribution Generalization [3.701112941066256]
  We exploit the independence for distribution generalization by taking into account higher moments. We prove that the proposed estimator is invariant to distributional shifts on the instruments. These results hold even in the under-identified case where the instruments are not sufficiently rich to identify the causal function.
  arXiv  Detail & Related papers  (2022-02-03T21:49:04Z)
• Dense Uncertainty Estimation via an Ensemble-based Conditional Latent Variable Model [68.34559610536614]
  We argue that the aleatoric uncertainty is an inherent attribute of the data and can only be correctly estimated with an unbiased oracle model. We propose a new sampling and selection strategy at train time to approximate the oracle model for aleatoric uncertainty estimation. Our results show that our solution achieves both accurate deterministic results and reliable uncertainty estimation.
  arXiv  Detail & Related papers  (2021-11-22T08:54:10Z)
• The Hidden Uncertainty in a Neural Networks Activations [105.4223982696279]
  The distribution of a neural network's latent representations has been successfully used to detect out-of-distribution (OOD) data. This work investigates whether this distribution correlates with a model's epistemic uncertainty, thus indicating its ability to generalise to novel inputs.
  arXiv  Detail & Related papers  (2020-12-05T17:30:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.