Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance

• URL: http://arxiv.org/abs/2006.14779v3
• Date: Tue, 12 Jan 2021 22:50:34 GMT
• Title: Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
• Authors: Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, Daniel S. Weld
• Abstract summary: Prior studies observed improvements from explanations only when the AI, alone, outperformed both the human and the best team. We conduct mixed-method user studies on three datasets, where an AI with accuracy comparable to humans helps participants solve a task. We find explanations increase the chance that humans will accept the AI's recommendation, regardless of its correctness.
• Score: 44.730580857733
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Many researchers motivate explainable AI with studies showing that human-AI team performance on decision-making tasks improves when the AI explains its recommendations. However, prior studies observed improvements from explanations only when the AI, alone, outperformed both the human and the best team. Can explanations help lead to complementary performance, where team accuracy is higher than either the human or the AI working solo? We conduct mixed-method user studies on three datasets, where an AI with accuracy comparable to humans helps participants solve a task (explaining itself in some conditions). While we observed complementary improvements from AI augmentation, they were not increased by explanations. Rather, explanations increased the chance that humans will accept the AI's recommendation, regardless of its correctness. Our result poses new challenges for human-centered AI: Can we develop explanatory approaches that encourage appropriate trust in AI, and therefore help generate (or improve) complementary performance?

Related papers

• Raising the Stakes: Performance Pressure Improves AI-Assisted Decision Making [57.53469908423318]
  We show the effects of performance pressure on AI advice reliance when laypeople complete a common AI-assisted task. We find that when the stakes are high, people use AI advice more appropriately than when stakes are lower, regardless of the presence of an AI explanation.
  arXiv  Detail & Related papers  (2024-10-21T22:39:52Z)
• Comparing Zealous and Restrained AI Recommendations in a Real-World Human-AI Collaboration Task [11.040918613968854]
  We argue that careful exploitation of the tradeoff between precision and recall can significantly improve team performance. We analyze the performance of 78 professional annotators working with a) no AI assistance, b) a high-precision "restrained" AI, and c) a high-recall "zealous" AI in over 3,466 person-hours of annotation work.
  arXiv  Detail & Related papers  (2024-10-06T23:19:19Z)
• Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making [5.381004207943597]
  We conducted an experiment with seven therapists and ten laypersons on the task of assessing post-stroke survivors' quality of motion. We analyzed their performance, agreement level on the task, and reliance on AI without and with two types of AI explanations. Our work discusses the potential of counterfactual explanations to better estimate the accuracy of an AI model and reduce over-reliance on wrong' AI outputs.
  arXiv  Detail & Related papers  (2023-08-08T16:23:46Z)
• Improving Human-AI Collaboration With Descriptions of AI Behavior [14.904401331154062]
  People work with AI systems to improve their decision making, but often under- or over-rely on AI predictions and perform worse than they would have unassisted. To help people appropriately rely on AI aids, we propose showing them behavior descriptions.
  arXiv  Detail & Related papers  (2023-01-06T00:33:08Z)
• Cybertrust: From Explainable to Actionable and Interpretable AI (AI2) [58.981120701284816]
  Actionable and Interpretable AI (AI2) will incorporate explicit quantifications and visualizations of user confidence in AI recommendations. It will allow examining and testing of AI system predictions to establish a basis for trust in the systems' decision making.
  arXiv  Detail & Related papers  (2022-01-26T18:53:09Z)
• The Who in XAI: How AI Background Shapes Perceptions of AI Explanations [61.49776160925216]
  We conduct a mixed-methods study of how two different groups--people with and without AI background--perceive different types of AI explanations. We find that (1) both groups showed unwarranted faith in numbers for different reasons and (2) each group found value in different explanations beyond their intended design.
  arXiv  Detail & Related papers  (2021-07-28T17:32:04Z)
• Trustworthy AI: A Computational Perspective [54.80482955088197]
  We focus on six of the most crucial dimensions in achieving trustworthy AI: (i) Safety & Robustness, (ii) Non-discrimination & Fairness, (iii) Explainability, (iv) Privacy, (v) Accountability & Auditability, and (vi) Environmental Well-Being. For each dimension, we review the recent related technologies according to a taxonomy and summarize their applications in real-world systems.
  arXiv  Detail & Related papers  (2021-07-12T14:21:46Z)
• To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making [4.877174544937129]
  People supported by AI-powered decision support tools frequently overrely on the AI. Adding explanations to the AI decisions does not appear to reduce the overreliance. Our research suggests that human cognitive motivation moderates the effectiveness of explainable AI solutions.
  arXiv  Detail & Related papers  (2021-02-19T00:38:53Z)
• Does Explainable Artificial Intelligence Improve Human Decision-Making? [17.18994675838646]
  We compare and evaluate objective human decision accuracy without AI (control), with an AI prediction (no explanation) and AI prediction with explanation. We find any kind of AI prediction tends to improve user decision accuracy, but no conclusive evidence that explainable AI has a meaningful impact. Our results indicate that, at least in some situations, the "why" information provided in explainable AI may not enhance user decision-making.
  arXiv  Detail & Related papers  (2020-06-19T15:46:13Z)
• Is the Most Accurate AI the Best Teammate? Optimizing AI for Teamwork [54.309495231017344]
  We argue that AI systems should be trained in a human-centered manner, directly optimized for team performance. We study this proposal for a specific type of human-AI teaming, where the human overseer chooses to either accept the AI recommendation or solve the task themselves. Our experiments with linear and non-linear models on real-world, high-stakes datasets show that the most accuracy AI may not lead to highest team performance.
  arXiv  Detail & Related papers  (2020-04-27T19:06:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.