On the Importance of Uncertainty in Decision-Making with Large Language Models

• URL: http://arxiv.org/abs/2404.02649v2
• Date: Sun, 14 Jul 2024 02:20:59 GMT
• Title: On the Importance of Uncertainty in Decision-Making with Large Language Models
• Authors: Nicolò Felicioni, Lucas Maystre, Sina Ghiassian, Kamil Ciosek,
• Abstract summary: We investigate the role of uncertainty in decision-making problems with natural language as input. We employ different techniques for uncertainty estimation, such as Laplace Approximation, Dropout, and Epinets.
• Score: 16.960086222920488
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We investigate the role of uncertainty in decision-making problems with natural language as input. For such tasks, using Large Language Models as agents has become the norm. However, none of the recent approaches employ any additional phase for estimating the uncertainty the agent has about the world during the decision-making task. We focus on a fundamental decision-making framework with natural language as input, which is the one of contextual bandits, where the context information consists of text. As a representative of the approaches with no uncertainty estimation, we consider an LLM bandit with a greedy policy, which picks the action corresponding to the largest predicted reward. We compare this baseline to LLM bandits that make active use of uncertainty estimation by integrating the uncertainty in a Thompson Sampling policy. We employ different techniques for uncertainty estimation, such as Laplace Approximation, Dropout, and Epinets. We empirically show on real-world data that the greedy policy performs worse than the Thompson Sampling policies. These findings suggest that, while overlooked in the LLM literature, uncertainty plays a fundamental role in bandit tasks with LLMs.

Related papers

• The Illusion of Certainty: Uncertainty quantification for LLMs fails under ambiguity [48.899855816199484]
  We introduce MAQA* and AmbigQA*, the first ambiguous question-answering (QA) datasets equipped with ground-truth answer distributions.<n>We show that predictive-distribution and ensemble-based estimators are fundamentally limited under ambiguity.
  arXiv  Detail & Related papers  (2025-11-06T14:46:35Z)
• Towards Reliable LLM-based Robot Planning via Combined Uncertainty Estimation [68.106428321492]
  Large language models (LLMs) demonstrate advanced reasoning abilities, enabling robots to understand natural language instructions and generate high-level plans with appropriate grounding.<n>LLMs hallucinations present a significant challenge, often leading to overconfident yet potentially misaligned or unsafe plans.<n>We present Combined Uncertainty estimation for Reliable Embodied planning (CURE), which decomposes the uncertainty into epistemic and intrinsic uncertainty, each estimated separately.
  arXiv  Detail & Related papers  (2025-10-09T10:26:58Z)
• Decoding Uncertainty: The Impact of Decoding Strategies for Uncertainty Estimation in Large Language Models [58.198220611190884]
  We investigate the impact of decoding strategies on uncertainty estimation in Large Language Models (LLMs)<n>Our experiments show that Contrastive Search, which mitigates repetition, yields better uncertainty estimates on average across a range of preference-aligned LLMs.
  arXiv  Detail & Related papers  (2025-09-20T13:48:13Z)
• Prospect Theory Fails for LLMs: Revealing Instability of Decision-Making under Epistemic Uncertainty [41.94502767947584]
  It remains largely unexplored whether Prospect Theory applies to contemporary Large Language Models.<n>We propose a more general and precise evaluation framework to model LLMs' decision-making behaviour under PT.<n>Our findings suggest that modelling LLMs' decision-making with PT is not consistently reliable.
  arXiv  Detail & Related papers  (2025-08-12T15:02:16Z)
• Position: Uncertainty Quantification Needs Reassessment for Large-language Model Agents [19.974114160377045]
  This position paper argues that this traditional dichotomy of uncertainties is too limited for the open and interactive setup that LLM agents operate in.<n>We propose three novel research directions that focus on uncertainties in such human-computer interactions.
  arXiv  Detail & Related papers  (2025-05-28T17:59:08Z)
• Token-Level Uncertainty Estimation for Large Language Model Reasoning [24.56760223952017]
  Large Language Models (LLMs) have demonstrated impressive capabilities, but their output quality remains inconsistent across various application scenarios.<n>We propose a token-level uncertainty estimation framework to enable LLMs to self-assess and self-improve their generation quality in mathematical reasoning.
  arXiv  Detail & Related papers  (2025-05-16T22:47:32Z)
• Uncertainty Quantification and Causal Considerations for Off-Policy Decision Making [4.514386953429771]
  Off-policy evaluation (OPE) seeks to assess the performance of a new policy using data collected under a different policy. Existing OPE methodologies suffer from several limitations arising from statistical uncertainty as well as causal considerations. We introduce the Marginal Ratio (MR) estimator, a novel OPE method that reduces variance by focusing on the marginal distribution of outcomes. Next, we propose Conformal Off-Policy Prediction (COPP), a principled approach for uncertainty quantification in OPE. Finally, we address causal unidentifiability in off-policy decision-making by developing novel bounds for sequential decision settings
  arXiv  Detail & Related papers  (2025-02-09T20:05:19Z)
• Unconditional Truthfulness: Learning Conditional Dependency for Uncertainty Quantification of Large Language Models [96.43562963756975]
  We train a regression model, which target variable is the gap between the conditional and the unconditional generation confidence. We use this learned conditional dependency model to modulate the uncertainty of the current generation step based on the uncertainty of the previous step.
  arXiv  Detail & Related papers  (2024-08-20T09:42:26Z)
• A Structured Review of Literature on Uncertainty in Machine Learning & Deep Learning [0.8667724053232616]
  We focus on a critical concern for adaptation of Machine Learning in risk-sensitive applications, namely understanding and quantifying uncertainty. Our paper approaches this topic in a structured way, providing a review of the literature in the various facets that uncertainty is enveloped in the ML process. Key contributions in this review are broadening the scope of uncertainty discussion, as well as an updated review of uncertainty quantification methods in Deep Learning.
  arXiv  Detail & Related papers  (2024-06-01T07:17:38Z)
• Semantic Density: Uncertainty Quantification for Large Language Models through Confidence Measurement in Semantic Space [14.715989394285238]
  Existing Large Language Models (LLMs) do not have an inherent functionality to provide the users with an uncertainty/confidence metric for each response it generates. A new framework is proposed in this paper to address these issues. Semantic density extracts uncertainty/confidence information for each response from a probability distribution perspective in semantic space.
  arXiv  Detail & Related papers  (2024-05-22T17:13:49Z)
• Uncertainty Estimation and Quantification for LLMs: A Simple Supervised Approach [6.209293868095268]
  We study the problem of uncertainty estimation and calibration for LLMs. We propose a supervised approach that leverages labeled datasets to estimate the uncertainty in LLMs' responses. Our method is easy to implement and adaptable to different levels of model accessibility including black box, grey box, and white box.
  arXiv  Detail & Related papers  (2024-04-24T17:10:35Z)
• Uncertainty in Language Models: Assessment through Rank-Calibration [65.10149293133846]
  Language Models (LMs) have shown promising performance in natural language generation. It is crucial to correctly quantify their uncertainty in responding to given inputs. We develop a novel and practical framework, termed $Rank$-$Calibration$, to assess uncertainty and confidence measures for LMs.
  arXiv  Detail & Related papers  (2024-04-04T02:31:05Z)
• Uncertainty Quantification for In-Context Learning of Large Language Models [52.891205009620364]
  In-context learning has emerged as a groundbreaking ability of Large Language Models (LLMs) We propose a novel formulation and corresponding estimation method to quantify both types of uncertainties. The proposed method offers an unsupervised way to understand the prediction of in-context learning in a plug-and-play fashion.
  arXiv  Detail & Related papers  (2024-02-15T18:46:24Z)
• Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling [69.83976050879318]
  In large language models (LLMs), identifying sources of uncertainty is an important step toward improving reliability, trustworthiness, and interpretability. In this paper, we introduce an uncertainty decomposition framework for LLMs, called input clarification ensembling. Our approach generates a set of clarifications for the input, feeds them into an LLM, and ensembles the corresponding predictions.
  arXiv  Detail & Related papers  (2023-11-15T05:58:35Z)
• Quantifying Uncertainty in Natural Language Explanations of Large Language Models [29.34960984639281]
  Large Language Models (LLMs) are increasingly used as powerful tools for high-stakes natural language processing (NLP) applications. We propose two novel metrics -- $textitVerbalized Uncertainty$ and $textitProbing Uncertainty$ -- to quantify the uncertainty of generated explanations. Our empirical analysis of benchmark datasets reveals that verbalized uncertainty is not a reliable estimate of explanation confidence.
  arXiv  Detail & Related papers  (2023-11-06T21:14:40Z)
• Uncertainty in Natural Language Processing: Sources, Quantification, and Applications [56.130945359053776]
  We provide a comprehensive review of uncertainty-relevant works in the NLP field. We first categorize the sources of uncertainty in natural language into three types, including input, system, and output. We discuss the challenges of uncertainty estimation in NLP and discuss potential future directions.
  arXiv  Detail & Related papers  (2023-06-05T06:46:53Z)
• Temporal Difference Uncertainties as a Signal for Exploration [76.6341354269013]
  An effective approach to exploration in reinforcement learning is to rely on an agent's uncertainty over the optimal policy. In this paper, we highlight that value estimates are easily biased and temporally inconsistent. We propose a novel method for estimating uncertainty over the value function that relies on inducing a distribution over temporal difference errors.
  arXiv  Detail & Related papers  (2020-10-05T18:11:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.