Diagnosing AI Explanation Methods with Folk Concepts of Behavior
- URL: http://arxiv.org/abs/2201.11239v5
- Date: Tue, 14 Nov 2023 11:32:11 GMT
- Title: Diagnosing AI Explanation Methods with Folk Concepts of Behavior
- Authors: Alon Jacovi, Jasmijn Bastings, Sebastian Gehrmann, Yoav Goldberg,
Katja Filippova
- Abstract summary: We consider "success" to depend not only on what information the explanation contains, but also on what information the human explainee understands from it.
We use folk concepts of behavior as a framework of social attribution by the human explainee.
- Score: 70.10183435379162
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We investigate a formalism for the conditions of a successful explanation of
AI. We consider "success" to depend not only on what information the
explanation contains, but also on what information the human explainee
understands from it. Theory of mind literature discusses the folk concepts that
humans use to understand and generalize behavior. We posit that folk concepts
of behavior provide us with a "language" that humans understand behavior with.
We use these folk concepts as a framework of social attribution by the human
explainee - the information constructs that humans are likely to comprehend
from explanations - by introducing a blueprint for an explanatory narrative
(Figure 1) that explains AI behavior with these constructs. We then demonstrate
that many XAI methods today can be mapped to folk concepts of behavior in a
qualitative evaluation. This allows us to uncover their failure modes that
prevent current methods from explaining successfully - i.e., the information
constructs that are missing for any given XAI method, and whose inclusion can
decrease the likelihood of misunderstanding AI behavior.
Related papers
- Explaining Explaining [0.882727051273924]
Explanation is key to people having confidence in high-stakes AI systems.
Machine-learning-based systems can't explain because they are usually black boxes.
We describe a hybrid approach to developing cognitive agents.
arXiv Detail & Related papers (2024-09-26T16:55:44Z) - Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development.
To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps.
These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
arXiv Detail & Related papers (2023-12-10T23:13:23Z) - Towards Human Cognition Level-based Experiment Design for Counterfactual
Explanations (XAI) [68.8204255655161]
The emphasis of XAI research appears to have turned to a more pragmatic explanation approach for better understanding.
An extensive area where cognitive science research may substantially influence XAI advancements is evaluating user knowledge and feedback.
We propose a framework to experiment with generating and evaluating the explanations on the grounds of different cognitive levels of understanding.
arXiv Detail & Related papers (2022-10-31T19:20:22Z) - Alterfactual Explanations -- The Relevance of Irrelevance for Explaining
AI Systems [0.9542023122304099]
We argue that in order to fully understand a decision, not only knowledge about relevant features is needed, but that the awareness of irrelevant information also highly contributes to the creation of a user's mental model of an AI system.
Our approach, which we call Alterfactual Explanations, is based on showing an alternative reality where irrelevant features of an AI's input are altered.
We show that alterfactual explanations are suited to convey an understanding of different aspects of the AI's reasoning than established counterfactual explanation methods.
arXiv Detail & Related papers (2022-07-19T16:20:37Z) - A psychological theory of explainability [5.715103211247915]
We propose a theory of how humans draw conclusions from saliency maps, the most common form of XAI explanation.
Our theory posits that absent explanation humans expect the AI to make similar decisions to themselves, and that they interpret an explanation by comparison to the explanations they themselves would give.
arXiv Detail & Related papers (2022-05-17T15:52:24Z) - CX-ToM: Counterfactual Explanations with Theory-of-Mind for Enhancing
Human Trust in Image Recognition Models [84.32751938563426]
We propose a new explainable AI (XAI) framework for explaining decisions made by a deep convolutional neural network (CNN)
In contrast to the current methods in XAI that generate explanations as a single shot response, we pose explanation as an iterative communication process.
Our framework generates sequence of explanations in a dialog by mediating the differences between the minds of machine and human user.
arXiv Detail & Related papers (2021-09-03T09:46:20Z) - The Who in XAI: How AI Background Shapes Perceptions of AI Explanations [61.49776160925216]
We conduct a mixed-methods study of how two different groups--people with and without AI background--perceive different types of AI explanations.
We find that (1) both groups showed unwarranted faith in numbers for different reasons and (2) each group found value in different explanations beyond their intended design.
arXiv Detail & Related papers (2021-07-28T17:32:04Z) - Mitigating belief projection in explainable artificial intelligence via
Bayesian Teaching [4.864819846886143]
Explainable AI (XAI) attempts to improve human understanding but rarely accounts for how people typically reason about unfamiliar agents.
We propose explicitly modeling the human explainee via Bayesian Teaching, which evaluates explanations by how much they shift explainees' inferences toward a desired goal.
arXiv Detail & Related papers (2021-02-07T21:23:24Z) - A general framework for scientifically inspired explanations in AI [76.48625630211943]
We instantiate the concept of structure of scientific explanation as the theoretical underpinning for a general framework in which explanations for AI systems can be implemented.
This framework aims to provide the tools to build a "mental-model" of any AI system so that the interaction with the user can provide information on demand and be closer to the nature of human-made explanations.
arXiv Detail & Related papers (2020-03-02T10:32:21Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.