Trust the AI, Doubt Yourself: The Effect of Urgency on Self-Confidence in Human-AI Interaction

• URL: http://arxiv.org/abs/2604.07535v1
• Date: Wed, 08 Apr 2026 19:17:59 GMT
• Title: Trust the AI, Doubt Yourself: The Effect of Urgency on Self-Confidence in Human-AI Interaction
• Abstract summary: Studies show that interactions with an AI system fosters trust in human users towards AI.<n>We show that although the presence of urgency in human-AI interactions does not affect the trust in AI, it may be detrimental to the human user's self-confidence and self-efficacy.
• Score: 7.401318863477374
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Studies show that interactions with an AI system fosters trust in human users towards AI. An often overlooked element of such interaction dynamics is the (sense of) urgency when the human user is prompted by an AI agent, e.g., for advice or guidance. In this paper, we show that although the presence of urgency in human-AI interactions does not affect the trust in AI, it may be detrimental to the human user's self-confidence and self-efficacy. In the long run, the loss of confidence may lead to performance loss, suboptimal decisions, human errors, and ultimately, unsustainable AI systems. Our evidence comes from an experiment with 30 human participants. Our results indicate that users may feel more confident in their work when they are eased into the human-AI setup rather than exposed to it without preparation. We elaborate on the implications of this finding for software engineers and decision-makers.

Related papers

• The Boundaries of Automation: A Theory of Persistent Human Participation [107.84677954791773]
  We argue that human participation may persist even with highly capable AI systems.<n>In some activities, the target is not fully specified in advance but instead emerges through the interaction itself.<n>This perspective has important implications for the limits of automation.
  arXiv  Detail & Related papers  (2026-07-23T17:33:43Z)
• Human-AI Complementarity: A Goal for Amplified Oversight [2.7005766101211663]
  This paper explores how we can leverage AI to improve the quality of human oversight.<n>We find that combining AI ratings and human ratings based on AI rater confidence is better than relying on either alone.
  arXiv  Detail & Related papers  (2025-10-30T14:11:52Z)
• Actionable AI: Enabling Non Experts to Understand and Configure AI Systems [5.534140394498714]
  Actionable AI allows non-experts to configure black-box agents.<n>In uncertain conditions, non-experts achieve good levels of performance.<n>We propose Actionable AI as a way to open access to AI-based agents.
  arXiv  Detail & Related papers  (2025-03-09T23:09:04Z)
• As Confidence Aligns: Exploring the Effect of AI Confidence on Human Self-confidence in Human-AI Decision Making [37.192236418976265]
  In human-AI decision-making, users' self-confidence aligns with AI confidence and such alignment can persist even after AI ceases to be involved.<n>The presence of real-time correctness feedback of decisions reduced the degree of alignment.
  arXiv  Detail & Related papers  (2025-01-22T13:25:14Z)
• How Performance Pressure Influences AI-Assisted Decision Making [52.997197698288936]
  We show how pressure and explainable AI (XAI) techniques interact with AI advice-taking behavior.<n>Our results show complex interaction effects, with different combinations of pressure and XAI techniques either improving or worsening AI advice taking behavior.
  arXiv  Detail & Related papers  (2024-10-21T22:39:52Z)
• Fairness in AI and Its Long-Term Implications on Society [68.8204255655161]
  We take a closer look at AI fairness and analyze how lack of AI fairness can lead to deepening of biases over time. We discuss how biased models can lead to more negative real-world outcomes for certain groups. If the issues persist, they could be reinforced by interactions with other risks and have severe implications on society in the form of social unrest.
  arXiv  Detail & Related papers  (2023-04-16T11:22:59Z)
• On the Influence of Explainable AI on Automation Bias [0.0]
  We aim to shed light on the potential to influence automation bias by explainable AI (XAI) We conduct an online experiment with regard to hotel review classifications and discuss first results.
  arXiv  Detail & Related papers  (2022-04-19T12:54:23Z)
• The Response Shift Paradigm to Quantify Human Trust in AI Recommendations [6.652641137999891]
  Explainability, interpretability and how much they affect human trust in AI systems are ultimately problems of human cognition as much as machine learning. We developed and validated a general purpose Human-AI interaction paradigm which quantifies the impact of AI recommendations on human decisions. Our proof-of-principle paradigm allows one to quantitatively compare the rapidly growing set of XAI/IAI approaches in terms of their effect on the end-user.
  arXiv  Detail & Related papers  (2022-02-16T22:02:09Z)
• Cybertrust: From Explainable to Actionable and Interpretable AI (AI2) [58.981120701284816]
  Actionable and Interpretable AI (AI2) will incorporate explicit quantifications and visualizations of user confidence in AI recommendations. It will allow examining and testing of AI system predictions to establish a basis for trust in the systems' decision making.
  arXiv  Detail & Related papers  (2022-01-26T18:53:09Z)
• Trustworthy AI: A Computational Perspective [54.80482955088197]
  We focus on six of the most crucial dimensions in achieving trustworthy AI: (i) Safety & Robustness, (ii) Non-discrimination & Fairness, (iii) Explainability, (iv) Privacy, (v) Accountability & Auditability, and (vi) Environmental Well-Being. For each dimension, we review the recent related technologies according to a taxonomy and summarize their applications in real-world systems.
  arXiv  Detail & Related papers  (2021-07-12T14:21:46Z)
• Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making [53.62514158534574]
  We study whether features that reveal case-specific model information can calibrate trust and improve the joint performance of the human and AI. We show that confidence score can help calibrate people's trust in an AI model, but trust calibration alone is not sufficient to improve AI-assisted decision making.
  arXiv  Detail & Related papers  (2020-01-07T15:33:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.