A Meta-Analysis on the Utility of Explainable Artificial Intelligence in Human-AI Decision-Making

• URL: http://arxiv.org/abs/2205.05126v1
• Date: Tue, 10 May 2022 19:08:10 GMT
• Title: A Meta-Analysis on the Utility of Explainable Artificial Intelligence in Human-AI Decision-Making
• Authors: Max Schemmer and Patrick Hemmer and Maximilian Nitsche and Niklas K\"uhl and Michael V\"ossing
• Abstract summary: We present an initial synthesis of existing research on XAI studies using a statistical meta-analysis. We observe a statistically positive impact of XAI on users' performance. We find no effect of explanations on users' performance compared to sole AI predictions.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Research in Artificial Intelligence (AI)-assisted decision-making is experiencing tremendous growth with a constantly rising number of studies evaluating the effect of AI with and without techniques from the field of explainable AI (XAI) on human decision-making performance. However, as tasks and experimental setups vary due to different objectives, some studies report improved user decision-making performance through XAI, while others report only negligible effects. Therefore, in this article, we present an initial synthesis of existing research on XAI studies using a statistical meta-analysis to derive implications across existing research. We observe a statistically positive impact of XAI on users' performance. Additionally, first results might indicate that human-AI decision-making yields better task performance on text data. However, we find no effect of explanations on users' performance compared to sole AI predictions. Our initial synthesis gives rise to future research to investigate the underlying causes as well as contribute to further development of algorithms that effectively benefit human decision-makers in the form of explanations.

Related papers

• Unexploited Information Value in Human-AI Collaboration [23.353778024330165]
  How to improve performance of a human-AI team is often not clear without knowing what particular information and strategies each agent employs. We propose a model based in statistically decision theory to analyze human-AI collaboration.
  arXiv  Detail & Related papers  (2024-11-03T01:34:45Z)
• Raising the Stakes: Performance Pressure Improves AI-Assisted Decision Making [57.53469908423318]
  We show the effects of performance pressure on AI advice reliance when laypeople complete a common AI-assisted task. We find that when the stakes are high, people use AI advice more appropriately than when stakes are lower, regardless of the presence of an AI explanation.
  arXiv  Detail & Related papers  (2024-10-21T22:39:52Z)
• Study on the Helpfulness of Explainable Artificial Intelligence [0.0]
  Legal, business, and ethical requirements motivate using effective XAI. We propose to evaluate XAI methods via the user's ability to successfully perform a proxy task. In other words, we address the helpfulness of XAI for human decision-making.
  arXiv  Detail & Related papers  (2024-10-14T14:03:52Z)
• Measuring Human Contribution in AI-Assisted Content Generation [68.03658922067487]
  This study raises the research question of measuring human contribution in AI-assisted content generation. By calculating mutual information between human input and AI-assisted output relative to self-information of AI-assisted output, we quantify the proportional information contribution of humans in content generation.
  arXiv  Detail & Related papers  (2024-08-27T05:56:04Z)
• Beyond Recommender: An Exploratory Study of the Effects of Different AI Roles in AI-Assisted Decision Making [48.179458030691286]
  We examine three AI roles: Recommender, Analyzer, and Devil's Advocate. Our results show each role's distinct strengths and limitations in task performance, reliance appropriateness, and user experience. These insights offer valuable implications for designing AI assistants with adaptive functional roles according to different situations.
  arXiv  Detail & Related papers  (2024-03-04T07:32:28Z)
• Explain To Decide: A Human-Centric Review on the Role of Explainable Artificial Intelligence in AI-assisted Decision Making [1.0878040851638]
  Machine learning models are error-prone and cannot be used autonomously. Explainable Artificial Intelligence (XAI) aids end-user understanding of the model. This paper surveyed the recent empirical studies on XAI's impact on human-AI decision-making.
  arXiv  Detail & Related papers  (2023-12-11T22:35:21Z)
• The Impact of Imperfect XAI on Human-AI Decision-Making [8.305869611846775]
  We evaluate how incorrect explanations influence humans' decision-making behavior in a bird species identification task. Our findings reveal the influence of imperfect XAI and humans' level of expertise on their reliance on AI and human-AI team performance.
  arXiv  Detail & Related papers  (2023-07-25T15:19:36Z)
• Impact Of Explainable AI On Cognitive Load: Insights From An Empirical Study [0.0]
  This study measures cognitive load, task performance, and task time for implementation-independent XAI explanation types using a COVID-19 use case. We found that these explanation types strongly influence end-users' cognitive load, task performance, and task time.
  arXiv  Detail & Related papers  (2023-04-18T09:52:09Z)
• The Role of AI in Drug Discovery: Challenges, Opportunities, and Strategies [97.5153823429076]
  The benefits, challenges and drawbacks of AI in this field are reviewed. The use of data augmentation, explainable AI, and the integration of AI with traditional experimental methods are also discussed.
  arXiv  Detail & Related papers  (2022-12-08T23:23:39Z)
• Connecting Algorithmic Research and Usage Contexts: A Perspective of Contextualized Evaluation for Explainable AI [65.44737844681256]
  A lack of consensus on how to evaluate explainable AI (XAI) hinders the advancement of the field. We argue that one way to close the gap is to develop evaluation methods that account for different user requirements.
  arXiv  Detail & Related papers  (2022-06-22T05:17:33Z)
• Counterfactual Explanations as Interventions in Latent Space [62.997667081978825]
  Counterfactual explanations aim to provide to end users a set of features that need to be changed in order to achieve a desired outcome. Current approaches rarely take into account the feasibility of actions needed to achieve the proposed explanations. We present Counterfactual Explanations as Interventions in Latent Space (CEILS), a methodology to generate counterfactual explanations.
  arXiv  Detail & Related papers  (2021-06-14T20:48:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.