Learning to Trust: How Humans Mentally Recalibrate AI Confidence Signals

• URL: http://arxiv.org/abs/2603.22634v1
• Date: Mon, 23 Mar 2026 23:23:25 GMT
• Title: Learning to Trust: How Humans Mentally Recalibrate AI Confidence Signals
• Abstract summary: We investigate whether humans can learn to mentally recalibrate AI confidence signals through repeated experience.<n>We present a computational model utilizing a linear-in-log-odds transformation and a Rescorla-Wagner learning rule to explain these dynamics.
• Score: 3.1153758106426603
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Productive human-AI collaboration requires appropriate reliance, yet contemporary AI systems are often miscalibrated, exhibiting systematic overconfidence or underconfidence. We investigate whether humans can learn to mentally recalibrate AI confidence signals through repeated experience. In a behavioral experiment (N = 200), participants predicted the AI's correctness across four AI calibration conditions: standard, overconfidence, underconfidence, and a counterintuitive "reverse confidence" mapping. Results demonstrate robust learning across all conditions, with participants significantly improving their accuracy, discrimination, and calibration alignment over 50 trials. We present a computational model utilizing a linear-in-log-odds (LLO) transformation and a Rescorla-Wagner learning rule to explain these dynamics. The model reveals that humans adapt by updating their baseline trust and confidence sensitivity, using asymmetric learning rates to prioritize the most informative errors. While humans can compensate for monotonic miscalibration, we identify a significant boundary in the reverse confidence scenario, where a substantial proportion of participants struggled to override initial inductive biases. These findings provide a mechanistic account of how humans adapt their trust in AI confidence signals through experience.

Related papers

• Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs [60.80435138270317]
  We apply reinforcement learning with metacognitive feedback to the problem of faithful calibration.<n>Experiments show RLMF achieves generalizable, state-of-the-art FC on diverse tasks.<n> RLMF surpasses standard RL by up to 63% while enhancing models' ability to assess and express their own capability limits.
  arXiv  Detail & Related papers  (2026-06-30T17:56:01Z)
• Metacognitive Sensitivity for Test-Time Dynamic Model Selection [0.0]
  We propose a new framework for evaluating and leveraging AI metacognition.<n>We introduce meta-d', a psychologically-grounded measure of metacognitive sensitivity, to characterise how reliably a model's confidence predicts its own accuracy.<n>We then use this dynamic sensitivity score as context for a bandit-based arbiter that performs test-time model selection.
  arXiv  Detail & Related papers  (2025-12-11T09:15:05Z)
• Guided Self-Evolving LLMs with Minimal Human Supervision [53.111086364268566]
  Unguided self-evolving systems often plateau quickly or degrade as training.<n>R-Few is a guided Self-Play Challenger acquirer framework that incorporates lightweight human oversight through in-context grounding and mixed training.<n>R-Few achieves consistent and iterative improvements across math and general reasoning benchmarks.
  arXiv  Detail & Related papers  (2025-12-02T07:06:11Z)
• ConfTuner: Training Large Language Models to Express Their Confidence Verbally [58.63318088243125]
  Large Language Models (LLMs) are increasingly deployed in high-stakes domains such as science, law, and healthcare.<n>LLMs are often observed to generate incorrect answers with high confidence, a phenomenon known as "overconfidence"
  arXiv  Detail & Related papers  (2025-08-26T09:25:32Z)
• Uncertainty Estimation by Human Perception versus Neural Models [7.273936646577467]
  Modern neural networks (NNs) often achieve high predictive accuracy but are poorly calibrated.<n>We investigate how human uncertainty compares to uncertainty estimated by NNs.
  arXiv  Detail & Related papers  (2025-06-18T20:00:20Z)
• Measurement of LLM's Philosophies of Human Nature [113.47929131143766]
  We design the standardized psychological scale specifically targeting large language models (LLM)<n>We show that current LLMs exhibit a systemic lack of trust in humans.<n>We propose a mental loop learning framework, which enables LLM to continuously optimize its value system.
  arXiv  Detail & Related papers  (2025-04-03T06:22:19Z)
• As Confidence Aligns: Exploring the Effect of AI Confidence on Human Self-confidence in Human-AI Decision Making [37.192236418976265]
  In human-AI decision-making, users' self-confidence aligns with AI confidence and such alignment can persist even after AI ceases to be involved.<n>The presence of real-time correctness feedback of decisions reduced the degree of alignment.
  arXiv  Detail & Related papers  (2025-01-22T13:25:14Z)
• Understanding the Effects of Miscalibrated AI Confidence on User Trust, Reliance, and Decision Efficacy [38.39755953750018]
  Miscalibrated AI confidence impairs users' appropriate reliance and reduces AI-assisted decision-making efficacy.<n>We find that communicating AI confidence calibration levels helps users to detect AI miscalibration.<n>However, since such communication decreases users' trust in uncalibrated AI, leading to high under-reliance, it does not improve the decision efficacy.
  arXiv  Detail & Related papers  (2024-02-12T13:16:30Z)
• A Diachronic Perspective on User Trust in AI under Uncertainty [52.44939679369428]
  Modern NLP systems are often uncalibrated, resulting in confidently incorrect predictions that undermine user trust. We study the evolution of user trust in response to trust-eroding events using a betting game.
  arXiv  Detail & Related papers  (2023-10-20T14:41:46Z)
• Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI [55.4046755826066]
  We discuss a model of trust inspired by, but not identical to, sociology's interpersonal trust (i.e., trust between people) We incorporate a formalization of 'contractual trust', such that trust between a user and an AI is trust that some implicit or explicit contract will hold. We discuss how to design trustworthy AI, how to evaluate whether trust has manifested, and whether it is warranted.
  arXiv  Detail & Related papers  (2020-10-15T03:07:23Z)
• Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making [53.62514158534574]
  We study whether features that reveal case-specific model information can calibrate trust and improve the joint performance of the human and AI. We show that confidence score can help calibrate people's trust in an AI model, but trust calibration alone is not sufficient to improve AI-assisted decision making.
  arXiv  Detail & Related papers  (2020-01-07T15:33:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.