Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missing

• URL: http://arxiv.org/abs/2507.10587v1
• Date: Fri, 11 Jul 2025 14:07:22 GMT
• Title: Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missing
• Authors: Dennis Ulmer, Alexandra Lorson, Ivan Titov, Christian Hardmeier,
• Abstract summary: We argue for anthropomimetic uncertainty, meaning that intuitive and trustworthy uncertainty communication requires a degree of linguistic authenticity and personalization to the user.<n>We conclude by pointing out unique factors in human-machine communication of uncertainty and deconstruct the data biases that influence machine uncertainty communication.
• Score: 66.04926909181653
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Human users increasingly rely on natural language interactions with large language models (LLMs) in order to receive help on a large variety of tasks and problems. However, the trustworthiness and perceived legitimacy of LLMs is undermined by the fact that their output is frequently stated in very confident terms, even when its accuracy is questionable. Therefore, there is a need to signal the confidence of the language model to a user in order to reap the benefits of human-machine collaboration and mitigate potential harms. Verbalized uncertainty is the expression of confidence with linguistic means, an approach that integrates perfectly into language-based interfaces. Nevertheless, most recent research in natural language processing (NLP) overlooks the nuances surrounding human uncertainty communication and the data biases that influence machine uncertainty communication. We argue for anthropomimetic uncertainty, meaning that intuitive and trustworthy uncertainty communication requires a degree of linguistic authenticity and personalization to the user, which could be achieved by emulating human communication. We present a thorough overview over the research in human uncertainty communication, survey ongoing research, and perform additional analyses to demonstrate so-far overlooked biases in verbalized uncertainty. We conclude by pointing out unique factors in human-machine communication of uncertainty and deconstruct anthropomimetic uncertainty into future research directions for NLP.

Related papers

• Spatial Language Likelihood Grounding Network for Bayesian Fusion of Human-Robot Observations [4.008130792416869]
  An uncertainty-aware fusion framework requires a grounded likelihood representing the uncertainty of human inputs.<n>This paper presents a Feature Pyramid Likelihood Grounding Network (FP-LGN) that grounds spatial language by learning relevant map image features.<n> Collaborative sensing results demonstrated that the grounded likelihood successfully enabled uncertainty-aware fusion of heterogeneous human language observations and robot sensor measurements.
  arXiv  Detail & Related papers  (2025-07-26T13:24:02Z)
• Uncertainty Estimation by Human Perception versus Neural Models [7.702016079410588]
  Modern neural networks (NNs) often achieve high predictive accuracy but remain poorly calibrated.<n>We investigate how human uncertainty compares to uncertainty estimated by NNs.
  arXiv  Detail & Related papers  (2025-06-18T20:00:20Z)
• Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinations [51.92795774118647]
  We find that verbal uncertainty'' is governed by a single linear feature in the representation space of LLMs.<n>We show that this has only moderate correlation with the actual semantic uncertainty'' of the model.
  arXiv  Detail & Related papers  (2025-03-18T17:51:04Z)
• Enhancing Trust in Large Language Models with Uncertainty-Aware Fine-Tuning [10.457661605916435]
  Large language models (LLMs) have revolutionized the field of natural language processing with their impressive reasoning and question-answering capabilities.<n>LLMs are sometimes prone to generating credible-sounding but incorrect information, a phenomenon known as hallucinations.<n>We introduce a novel uncertainty-aware causal language modeling loss function, grounded in the principles of decision theory.
  arXiv  Detail & Related papers  (2024-12-03T23:14:47Z)
• Trustworthy Alignment of Retrieval-Augmented Large Language Models via Reinforcement Learning [84.94709351266557]
  We focus on the trustworthiness of language models with respect to retrieval augmentation. We deem that retrieval-augmented language models have the inherent capabilities of supplying response according to both contextual and parametric knowledge. Inspired by aligning language models with human preference, we take the first step towards aligning retrieval-augmented language models to a status where it responds relying merely on the external evidence.
  arXiv  Detail & Related papers  (2024-10-22T09:25:21Z)
• Multimodal Fusion with LLMs for Engagement Prediction in Natural Conversation [70.52558242336988]
  We focus on predicting engagement in dyadic interactions by scrutinizing verbal and non-verbal cues, aiming to detect signs of disinterest or confusion. In this work, we collect a dataset featuring 34 participants engaged in casual dyadic conversations, each providing self-reported engagement ratings at the end of each conversation. We introduce a novel fusion strategy using Large Language Models (LLMs) to integrate multiple behavior modalities into a multimodal transcript''
  arXiv  Detail & Related papers  (2024-09-13T18:28:12Z)
• Learning and communication pressures in neural networks: Lessons from emergent communication [5.371337604556311]
  We look at three cases where mismatches between the emergent linguistic behavior of neural agents and humans were resolved.<n>We identify key pressures at play for language learning and emergence: communicative success, production effort, learnability, and other psycho-/sociolinguistic factors.
  arXiv  Detail & Related papers  (2024-03-21T14:33:34Z)
• Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertainty [53.336235704123915]
  We investigate how LMs incorporate confidence in responses via natural language and how downstream users behave in response to LM-articulated uncertainties. We find that LMs are reluctant to express uncertainties when answering questions even when they produce incorrect responses. We test the risks of LM overconfidence by conducting human experiments and show that users rely heavily on LM generations. Lastly, we investigate the preference-annotated datasets used in post training alignment and find that humans are biased against texts with uncertainty.
  arXiv  Detail & Related papers  (2024-01-12T18:03:30Z)
• Uncertainty in Natural Language Processing: Sources, Quantification, and Applications [56.130945359053776]
  We provide a comprehensive review of uncertainty-relevant works in the NLP field. We first categorize the sources of uncertainty in natural language into three types, including input, system, and output. We discuss the challenges of uncertainty estimation in NLP and discuss potential future directions.
  arXiv  Detail & Related papers  (2023-06-05T06:46:53Z)
• Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Models [74.07684768317705]
  LMs are highly sensitive to markers of certainty in prompts, with accuies varying more than 80%. We find that expressions of high certainty result in a decrease in accuracy as compared to low expressions; similarly, factive verbs hurt performance, while evidentials benefit performance. These associations may suggest that LMs is based on observed language use, rather than truly reflecting uncertainty.
  arXiv  Detail & Related papers  (2023-02-26T23:46:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.