A Human-Centered Interpretability Framework Based on Weight of Evidence

• URL: http://arxiv.org/abs/2104.13299v1
• Date: Tue, 27 Apr 2021 16:13:35 GMT
• Title: A Human-Centered Interpretability Framework Based on Weight of Evidence
• Authors: David Alvarez-Melis, Harmanpreet Kaur, Hal Daum\'e III, Hanna Wallach, Jennifer Wortman Vaughan
• Abstract summary: We take a human-centered approach to interpretable machine learning. We propose a list of design principles for machine-generated explanations meaningful to humans. We show that this method can be adapted to handle high-dimensional, multi-class settings.
• Score: 26.94750208505883
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we take a human-centered approach to interpretable machine learning. First, drawing inspiration from the study of explanation in philosophy, cognitive science, and the social sciences, we propose a list of design principles for machine-generated explanations that are meaningful to humans. Using the concept of weight of evidence from information theory, we develop a method for producing explanations that adhere to these principles. We show that this method can be adapted to handle high-dimensional, multi-class settings, yielding a flexible meta-algorithm for generating explanations. We demonstrate that these explanations can be estimated accurately from finite samples and are robust to small perturbations of the inputs. We also evaluate our method through a qualitative user study with machine learning practitioners, where we observe that the resulting explanations are usable despite some participants struggling with background concepts like prior class probabilities. Finally, we conclude by surfacing design implications for interpretability tools

Related papers

• Concept Induction using LLMs: a user experiment for assessment [1.1982127665424676]
  This study explores the potential of a Large Language Model (LLM) to generate high-level concepts that are meaningful as explanations for humans. We compare the concepts generated by the LLM with two other methods: concepts generated by humans and the ECII concept induction system. Our findings indicate that while human-generated explanations remain superior, concepts derived from GPT-4 are more comprehensible to humans compared to those generated by ECII.
  arXiv  Detail & Related papers  (2024-04-18T03:22:02Z)
• Estimation of Concept Explanations Should be Uncertainty Aware [39.598213804572396]
  We study a specific kind called Concept Explanations, where the goal is to interpret a model using human-understandable concepts. Although popular for their easy interpretation, concept explanations are known to be noisy. We propose an uncertainty-aware Bayesian estimation method to address these issues, which readily improved the quality of explanations.
  arXiv  Detail & Related papers  (2023-12-13T11:17:27Z)
• Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
  We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development. To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps. These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
  arXiv  Detail & Related papers  (2023-12-10T23:13:23Z)
• Interpreting Pretrained Language Models via Concept Bottlenecks [55.47515772358389]
  Pretrained language models (PLMs) have made significant strides in various natural language processing tasks. The lack of interpretability due to their black-box'' nature poses challenges for responsible implementation. We propose a novel approach to interpreting PLMs by employing high-level, meaningful concepts that are easily understandable for humans.
  arXiv  Detail & Related papers  (2023-11-08T20:41:18Z)
• Explaining Explainability: Towards Deeper Actionable Insights into Deep Learning through Second-order Explainability [70.60433013657693]
  Second-order explainable AI (SOXAI) was recently proposed to extend explainable AI (XAI) from the instance level to the dataset level. We demonstrate for the first time, via example classification and segmentation cases, that eliminating irrelevant concepts from the training set based on actionable insights from SOXAI can enhance a model's performance.
  arXiv  Detail & Related papers  (2023-06-14T23:24:01Z)
• Overlooked factors in concept-based explanations: Dataset choice, concept learnability, and human capability [25.545486537295144]
  Concept-based interpretability methods aim to explain deep neural network model predictions using a predefined set of semantic concepts. Despite their popularity, they suffer from limitations that are not well-understood and articulated by the literature. We analyze three commonly overlooked factors in concept-based explanations.
  arXiv  Detail & Related papers  (2022-07-20T01:59:39Z)
• From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagation [16.783836191022445]
  The field of eXplainable Artificial Intelligence (XAI) aims to bring transparency to today's powerful but opaque deep learning models. While local XAI methods explain individual predictions in form of attribution maps, global explanation techniques visualize what concepts a model has generally learned to encode.
  arXiv  Detail & Related papers  (2022-06-07T12:05:58Z)
• Human-Centered Concept Explanations for Neural Networks [47.71169918421306]
  We introduce concept explanations including the class of Concept Activation Vectors (CAV) We then discuss approaches to automatically extract concepts, and approaches to address some of their caveats. Finally, we discuss some case studies that showcase the utility of such concept-based explanations in synthetic settings and real world applications.
  arXiv  Detail & Related papers  (2022-02-25T01:27:31Z)
• Formalising Concepts as Grounded Abstractions [68.24080871981869]
  This report shows how representation learning can be used to induce concepts from raw data. The main technical goal of this report is to show how techniques from representation learning can be married with a lattice-theoretic formulation of conceptual spaces.
  arXiv  Detail & Related papers  (2021-01-13T15:22:01Z)
• A general framework for scientifically inspired explanations in AI [76.48625630211943]
  We instantiate the concept of structure of scientific explanation as the theoretical underpinning for a general framework in which explanations for AI systems can be implemented. This framework aims to provide the tools to build a "mental-model" of any AI system so that the interaction with the user can provide information on demand and be closer to the nature of human-made explanations.
  arXiv  Detail & Related papers  (2020-03-02T10:32:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.