Black Box Model Explanations and the Human Interpretability Expectations -- An Analysis in the Context of Homicide Prediction

• URL: http://arxiv.org/abs/2210.10849v2
• Date: Thu, 4 Jul 2024 12:39:08 GMT
• Title: Black Box Model Explanations and the Human Interpretability Expectations -- An Analysis in the Context of Homicide Prediction
• Authors: José Ribeiro, Níkolas Carneiro, Ronnie Alves,
• Abstract summary: Strategies based on Explainable Artificial Intelligence (XAI) have promoted better human interpretability of the results of black box models. This research addresses a real-world classification problem related to homicide prediction. It used 6 different XAI methods to generate explanations and 6 different human experts.
• Score: 0.5898893619901381
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Strategies based on Explainable Artificial Intelligence (XAI) have promoted better human interpretability of the results of black box models. This opens up the possibility of questioning whether explanations created by XAI methods meet human expectations. The XAI methods being currently used (Ciu, Dalex, Eli5, Lofo, Shap, and Skater) provide various forms of explanations, including global rankings of relevance of features, which allow for an overview of how the model is explained as a result of its inputs and outputs. These methods provide for an increase in the explainability of the model and a greater interpretability grounded on the context of the problem. Intending to shed light on the explanations generated by XAI methods and their interpretations, this research addresses a real-world classification problem related to homicide prediction, already peer-validated, replicated its proposed black box model and used 6 different XAI methods to generate explanations and 6 different human experts. The results were generated through calculations of correlations, comparative analysis and identification of relationships between all ranks of features produced. It was found that even though it is a model that is difficult to explain, 75\% of the expectations of human experts were met, with approximately 48\% agreement between results from XAI methods and human experts. The results allow for answering questions such as: "Are the Expectation of Interpretation generated among different human experts similar?", "Do the different XAI methods generate similar explanations for the proposed problem?", "Can explanations generated by XAI methods meet human expectation of Interpretations?", and "Can Explanations and Expectations of Interpretation work together?".

Related papers

• How Reliable and Stable are Explanations of XAI Methods? [0.4749981032986242]
  Black box models are increasingly being used in the daily lives of human beings living in society. There has been the emergence of Explainable Artificial Intelligence (XAI) methods aimed at generating additional explanations regarding how the model makes certain predictions. It was found that current XAI methods are sensitive to perturbations, with the exception of one specific method.
  arXiv  Detail & Related papers  (2024-07-03T13:47:41Z)
• Unraveling the Dilemma of AI Errors: Exploring the Effectiveness of Human and Machine Explanations for Large Language Models [8.863857300695667]
  We analyzed 156 human-generated text and saliency-based explanations in a question-answering task. Our findings show that participants found human saliency maps to be more helpful in explaining AI answers than machine saliency maps. This finding hints at the dilemma of AI errors in explanation, where helpful explanations can lead to lower task performance when they support wrong AI predictions.
  arXiv  Detail & Related papers  (2024-04-11T13:16:51Z)
• Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
  We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development. To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps. These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
  arXiv  Detail & Related papers  (2023-12-10T23:13:23Z)
• Explaining Explainability: Towards Deeper Actionable Insights into Deep Learning through Second-order Explainability [70.60433013657693]
  Second-order explainable AI (SOXAI) was recently proposed to extend explainable AI (XAI) from the instance level to the dataset level. We demonstrate for the first time, via example classification and segmentation cases, that eliminating irrelevant concepts from the training set based on actionable insights from SOXAI can enhance a model's performance.
  arXiv  Detail & Related papers  (2023-06-14T23:24:01Z)
• Diagnosing AI Explanation Methods with Folk Concepts of Behavior [70.10183435379162]
  We consider "success" to depend not only on what information the explanation contains, but also on what information the human explainee understands from it. We use folk concepts of behavior as a framework of social attribution by the human explainee.
  arXiv  Detail & Related papers  (2022-01-27T00:19:41Z)
• CX-ToM: Counterfactual Explanations with Theory-of-Mind for Enhancing Human Trust in Image Recognition Models [84.32751938563426]
  We propose a new explainable AI (XAI) framework for explaining decisions made by a deep convolutional neural network (CNN) In contrast to the current methods in XAI that generate explanations as a single shot response, we pose explanation as an iterative communication process. Our framework generates sequence of explanations in a dialog by mediating the differences between the minds of machine and human user.
  arXiv  Detail & Related papers  (2021-09-03T09:46:20Z)
• The Who in XAI: How AI Background Shapes Perceptions of AI Explanations [61.49776160925216]
  We conduct a mixed-methods study of how two different groups--people with and without AI background--perceive different types of AI explanations. We find that (1) both groups showed unwarranted faith in numbers for different reasons and (2) each group found value in different explanations beyond their intended design.
  arXiv  Detail & Related papers  (2021-07-28T17:32:04Z)
• A Turing Test for Transparency [0.0]
  A central goal of explainable artificial intelligence (XAI) is to improve the trust relationship in human-AI interaction. Recent empirical evidence shows that explanations can have the opposite effect. This effect challenges the very goal of XAI and implies that responsible usage of transparent AI methods has to consider the ability of humans to distinguish machine generated from human explanations.
  arXiv  Detail & Related papers  (2021-06-21T20:09:40Z)
• Argumentative XAI: A Survey [15.294433619347082]
  We overview XAI approaches built using methods from the field of computational argumentation. We focus on different types of explanation (intrinsic and post-hoc), different models with which argumentation-based explanations are deployed, different forms of delivery, and different argumentation frameworks they use.
  arXiv  Detail & Related papers  (2021-05-24T13:32:59Z)
• Interpretable Deep Learning: Interpretations, Interpretability, Trustworthiness, and Beyond [49.93153180169685]
  We introduce and clarify two basic concepts-interpretations and interpretability-that people usually get confused. We elaborate the design of several recent interpretation algorithms, from different perspectives, through proposing a new taxonomy. We summarize the existing work in evaluating models' interpretability using "trustworthy" interpretation algorithms.
  arXiv  Detail & Related papers  (2021-03-19T08:40:30Z)
• Challenging common interpretability assumptions in feature attribution explanations [0.0]
  We empirically evaluate the veracity of three common interpretability assumptions through a large scale human-subjects experiment. We find that feature attribution explanations provide marginal utility in our task for a human decision maker.
  arXiv  Detail & Related papers  (2020-12-04T17:57:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.