A psychological theory of explainability

• URL: http://arxiv.org/abs/2205.08452v1
• Date: Tue, 17 May 2022 15:52:24 GMT
• Title: A psychological theory of explainability
• Authors: Scott Cheng-Hsin Yang, Tomas Folke, Patrick Shafto
• Abstract summary: We propose a theory of how humans draw conclusions from saliency maps, the most common form of XAI explanation. Our theory posits that absent explanation humans expect the AI to make similar decisions to themselves, and that they interpret an explanation by comparison to the explanations they themselves would give.
• Score: 5.715103211247915
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The goal of explainable Artificial Intelligence (XAI) is to generate human-interpretable explanations, but there are no computationally precise theories of how humans interpret AI generated explanations. The lack of theory means that validation of XAI must be done empirically, on a case-by-case basis, which prevents systematic theory-building in XAI. We propose a psychological theory of how humans draw conclusions from saliency maps, the most common form of XAI explanation, which for the first time allows for precise prediction of explainee inference conditioned on explanation. Our theory posits that absent explanation humans expect the AI to make similar decisions to themselves, and that they interpret an explanation by comparison to the explanations they themselves would give. Comparison is formalized via Shepard's universal law of generalization in a similarity space, a classic theory from cognitive science. A pre-registered user study on AI image classifications with saliency map explanations demonstrate that our theory quantitatively matches participants' predictions of the AI.

Related papers

• Axe the X in XAI: A Plea for Understandable AI [0.0]
  I argue that the notion of explainability as it is currently used in the XAI literature bears little resemblance to the traditional concept of scientific explanation. It would be more fruitful to use the label "understandable AI" to avoid the confusion that surrounds the goal and purposes of XAI.
  arXiv  Detail & Related papers  (2024-03-01T06:28:53Z)
• A Theoretical Framework for AI Models Explainability with Application in Biomedicine [3.5742391373143474]
  We propose a novel definition of explanation that is a synthesis of what can be found in the literature. We fit explanations into the properties of faithfulness (i.e., the explanation being a true description of the model's inner workings and decision-making process) and plausibility (i.e., how much the explanation looks convincing to the user)
  arXiv  Detail & Related papers  (2022-12-29T20:05:26Z)
• Towards Human Cognition Level-based Experiment Design for Counterfactual Explanations (XAI) [68.8204255655161]
  The emphasis of XAI research appears to have turned to a more pragmatic explanation approach for better understanding. An extensive area where cognitive science research may substantially influence XAI advancements is evaluating user knowledge and feedback. We propose a framework to experiment with generating and evaluating the explanations on the grounds of different cognitive levels of understanding.
  arXiv  Detail & Related papers  (2022-10-31T19:20:22Z)
• Eliminating The Impossible, Whatever Remains Must Be True [46.39428193548396]
  We show how one can apply background knowledge to give more succinct "why" formal explanations. We also show how to use existing rule induction techniques to efficiently extract background information from a dataset.
  arXiv  Detail & Related papers  (2022-06-20T03:18:14Z)
• Diagnosing AI Explanation Methods with Folk Concepts of Behavior [70.10183435379162]
  We consider "success" to depend not only on what information the explanation contains, but also on what information the human explainee understands from it. We use folk concepts of behavior as a framework of social attribution by the human explainee.
  arXiv  Detail & Related papers  (2022-01-27T00:19:41Z)
• Towards Relatable Explainable AI with the Perceptual Process [5.581885362337179]
  We argue that explanations must be more relatable to other concepts, hypotheticals, and associations. Inspired by cognitive psychology, we propose the XAI Perceptual Processing Framework and RexNet model for relatable explainable AI.
  arXiv  Detail & Related papers  (2021-12-28T05:48:53Z)
• CX-ToM: Counterfactual Explanations with Theory-of-Mind for Enhancing Human Trust in Image Recognition Models [84.32751938563426]
  We propose a new explainable AI (XAI) framework for explaining decisions made by a deep convolutional neural network (CNN) In contrast to the current methods in XAI that generate explanations as a single shot response, we pose explanation as an iterative communication process. Our framework generates sequence of explanations in a dialog by mediating the differences between the minds of machine and human user.
  arXiv  Detail & Related papers  (2021-09-03T09:46:20Z)
• The Who in XAI: How AI Background Shapes Perceptions of AI Explanations [61.49776160925216]
  We conduct a mixed-methods study of how two different groups--people with and without AI background--perceive different types of AI explanations. We find that (1) both groups showed unwarranted faith in numbers for different reasons and (2) each group found value in different explanations beyond their intended design.
  arXiv  Detail & Related papers  (2021-07-28T17:32:04Z)
• Explainable AI without Interpretable Model [0.0]
  It has become more important than ever that AI systems would be able to explain the reasoning behind their results to end-users. Most Explainable AI (XAI) methods are based on extracting an interpretable model that can be used for producing explanations. The notions of Contextual Importance and Utility (CIU) presented in this paper make it possible to produce human-like explanations of black-box outcomes directly.
  arXiv  Detail & Related papers  (2020-09-29T13:29:44Z)
• A general framework for scientifically inspired explanations in AI [76.48625630211943]
  We instantiate the concept of structure of scientific explanation as the theoretical underpinning for a general framework in which explanations for AI systems can be implemented. This framework aims to provide the tools to build a "mental-model" of any AI system so that the interaction with the user can provide information on demand and be closer to the nature of human-made explanations.
  arXiv  Detail & Related papers  (2020-03-02T10:32:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.