Accuracy-Time Tradeoffs in AI-Assisted Decision Making under Time Pressure

• URL: http://arxiv.org/abs/2306.07458v3
• Date: Sun, 11 Feb 2024 16:22:40 GMT
• Title: Accuracy-Time Tradeoffs in AI-Assisted Decision Making under Time Pressure
• Abstract summary: We show how time pressure affects how users use different AI assistances. We also find that a user's overreliance rate is a key predictor of their behaviour. Overall, our work suggests that AI assistances have different accuracy-time tradeoffs when people are under time pressure compared to no time pressure.
• Score: 31.723686787975687
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In settings where users both need high accuracy and are time-pressured, such as doctors working in emergency rooms, we want to provide AI assistance that both increases decision accuracy and reduces decision-making time. Current literature focusses on how users interact with AI assistance when there is no time pressure, finding that different AI assistances have different benefits: some can reduce time taken while increasing overreliance on AI, while others do the opposite. The precise benefit can depend on both the user and task. In time-pressured scenarios, adapting when we show AI assistance is especially important: relying on the AI assistance can save time, and can therefore be beneficial when the AI is likely to be right. We would ideally adapt what AI assistance we show depending on various properties (of the task and of the user) in order to best trade off accuracy and time. We introduce a study where users have to answer a series of logic puzzles. We find that time pressure affects how users use different AI assistances, making some assistances more beneficial than others when compared to no-time-pressure settings. We also find that a user's overreliance rate is a key predictor of their behaviour: overreliers and not-overreliers use different AI assistance types differently. We find marginal correlations between a user's overreliance rate (which is related to the user's trust in AI recommendations) and their personality traits (Big Five Personality traits). Overall, our work suggests that AI assistances have different accuracy-time tradeoffs when people are under time pressure compared to no time pressure, and we explore how we might adapt AI assistances in this setting.

Related papers

• How AI Assistance Affects Human Skill Development: A Study of Learning with Logic Puzzles [12.810110526024873]
  We examine tension in a controlled logic-puzzle experiment involving on-demand AI assistance.<n>We find that lower-cost assistance induces more frequent AI use.<n>We also find that participants who request AI assistance during the AI-access phase perform worse at the task after assistance is removed.
  arXiv  Detail & Related papers  (2026-08-24T17:46:11Z)
• The efficiency-gain illusion: People underestimate the rate of AI use and overestimate its benefits on simple tasks [39.088619511248986]
  We investigate people's propensity to use AI for cognitively simple tasks and assess whether their reliance is well-calibrated.<n>We find that people frequently choose to use AI even when doing so is inefficient.<n>Our results shed light on the mechanisms and biases underlying people's choice of whether to use AI as well as the risk of an overreliance feedback loop.
  arXiv  Detail & Related papers  (2026-05-21T16:28:20Z)
• The Impact of AI Usage and Informativeness on Skill Development in Logical Reasoning [13.546638788752924]
  We examine how both AI usage and informativeness can shape learning in a logical reasoning task with on-demand access to AI assistance.<n>We find that greater AI usage is associated with weaker skill development, whereas light AI users perform similarly to matched users who do not use AI.<n>Our findings suggest that AI can, depending on context, either complement human skill development by amplifying independent reasoning or can act as a substitute.
  arXiv  Detail & Related papers  (2026-05-20T19:55:57Z)
• Explainable AI as a Double-Edged Sword in Dermatology: The Impact on Clinicians versus The Public [46.86429592892395]
  explainable AI (XAI) addresses this by providing AI decision-making insight.<n>We present results from two large-scale experiments combining a fairness-based diagnosis AI model and different XAI explanations.
  arXiv  Detail & Related papers  (2025-12-14T00:06:06Z)
• Raising the Stakes: Performance Pressure Improves AI-Assisted Decision Making [57.53469908423318]
  We show the effects of performance pressure on AI advice reliance when laypeople complete a common AI-assisted task. We find that when the stakes are high, people use AI advice more appropriately than when stakes are lower, regardless of the presence of an AI explanation.
  arXiv  Detail & Related papers  (2024-10-21T22:39:52Z)
• Comparing Zealous and Restrained AI Recommendations in a Real-World Human-AI Collaboration Task [11.040918613968854]
  We argue that careful exploitation of the tradeoff between precision and recall can significantly improve team performance. We analyze the performance of 78 professional annotators working with a) no AI assistance, b) a high-precision "restrained" AI, and c) a high-recall "zealous" AI in over 3,466 person-hours of annotation work.
  arXiv  Detail & Related papers  (2024-10-06T23:19:19Z)
• The Ethics of Advanced AI Assistants [53.89899371095332]
  This paper focuses on the opportunities and the ethical and societal risks posed by advanced AI assistants. We define advanced AI assistants as artificial agents with natural language interfaces, whose function is to plan and execute sequences of actions on behalf of a user. We consider the deployment of advanced assistants at a societal scale, focusing on cooperation, equity and access, misinformation, economic impact, the environment and how best to evaluate advanced AI assistants.
  arXiv  Detail & Related papers  (2024-04-24T23:18:46Z)
• The Role of Heuristics and Biases During Complex Choices with an AI Teammate [0.0]
  We argue that classic experimental methods are insufficient for studying complex choices made with AI helpers. We show that framing and anchoring effects impact how people work with an AI helper and are predictive of choice outcomes.
  arXiv  Detail & Related papers  (2023-01-14T20:06:43Z)
• Improving Human-AI Collaboration With Descriptions of AI Behavior [14.904401331154062]
  People work with AI systems to improve their decision making, but often under- or over-rely on AI predictions and perform worse than they would have unassisted. To help people appropriately rely on AI aids, we propose showing them behavior descriptions.
  arXiv  Detail & Related papers  (2023-01-06T00:33:08Z)
• Cybertrust: From Explainable to Actionable and Interpretable AI (AI2) [58.981120701284816]
  Actionable and Interpretable AI (AI2) will incorporate explicit quantifications and visualizations of user confidence in AI recommendations. It will allow examining and testing of AI system predictions to establish a basis for trust in the systems' decision making.
  arXiv  Detail & Related papers  (2022-01-26T18:53:09Z)
• Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance [44.730580857733]
  Prior studies observed improvements from explanations only when the AI, alone, outperformed both the human and the best team. We conduct mixed-method user studies on three datasets, where an AI with accuracy comparable to humans helps participants solve a task. We find explanations increase the chance that humans will accept the AI's recommendation, regardless of its correctness.
  arXiv  Detail & Related papers  (2020-06-26T03:34:04Z)
• Is the Most Accurate AI the Best Teammate? Optimizing AI for Teamwork [54.309495231017344]
  We argue that AI systems should be trained in a human-centered manner, directly optimized for team performance. We study this proposal for a specific type of human-AI teaming, where the human overseer chooses to either accept the AI recommendation or solve the task themselves. Our experiments with linear and non-linear models on real-world, high-stakes datasets show that the most accuracy AI may not lead to highest team performance.
  arXiv  Detail & Related papers  (2020-04-27T19:06:28Z)
• Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making [53.62514158534574]
  We study whether features that reveal case-specific model information can calibrate trust and improve the joint performance of the human and AI. We show that confidence score can help calibrate people's trust in an AI model, but trust calibration alone is not sufficient to improve AI-assisted decision making.
  arXiv  Detail & Related papers  (2020-01-07T15:33:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.