Less is More: Discovering Concise Network Explanations

• URL: http://arxiv.org/abs/2405.15243v3
• Date: Mon, 8 Jul 2024 14:13:17 GMT
• Title: Less is More: Discovering Concise Network Explanations
• Authors: Neehar Kondapaneni, Markus Marks, Oisin Mac Aodha, Pietro Perona,
• Abstract summary: We introduce Discovering Conceptual Network Explanations (DCNE), a new approach for generating human-comprehensible visual explanations. Our method automatically finds visual explanations that are critical for discriminating between classes. DCNE represents a step forward in making neural network decisions accessible and interpretable to humans.
• Score: 26.126343100127936
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce Discovering Conceptual Network Explanations (DCNE), a new approach for generating human-comprehensible visual explanations to enhance the interpretability of deep neural image classifiers. Our method automatically finds visual explanations that are critical for discriminating between classes. This is achieved by simultaneously optimizing three criteria: the explanations should be few, diverse, and human-interpretable. Our approach builds on the recently introduced Concept Relevance Propagation (CRP) explainability method. While CRP is effective at describing individual neuronal activations, it generates too many concepts, which impacts human comprehension. Instead, DCNE selects the few most important explanations. We introduce a new evaluation dataset centered on the challenging task of classifying birds, enabling us to compare the alignment of DCNE's explanations to those of human expert-defined ones. Compared to existing eXplainable Artificial Intelligence (XAI) methods, DCNE has a desirable trade-off between conciseness and completeness when summarizing network explanations. It produces 1/30 of CRP's explanations while only resulting in a slight reduction in explanation quality. DCNE represents a step forward in making neural network decisions accessible and interpretable to humans, providing a valuable tool for both researchers and practitioners in XAI and model alignment.

Related papers

• On the Value of Labeled Data and Symbolic Methods for Hidden Neuron Activation Analysis [1.55858752644861]
  State of the art indicates that hidden node activations can, in some cases, be interpretable in a way that makes sense to humans. We introduce a novel model-agnostic post-hoc Explainable AI method demonstrating that it provides meaningful interpretations.
  arXiv  Detail & Related papers  (2024-04-21T07:57:45Z)
• Concept Induction using LLMs: a user experiment for assessment [1.1982127665424676]
  This study explores the potential of a Large Language Model (LLM) to generate high-level concepts that are meaningful as explanations for humans. We compare the concepts generated by the LLM with two other methods: concepts generated by humans and the ECII concept induction system. Our findings indicate that while human-generated explanations remain superior, concepts derived from GPT-4 are more comprehensible to humans compared to those generated by ECII.
  arXiv  Detail & Related papers  (2024-04-18T03:22:02Z)
• Everybody Needs a Little HELP: Explaining Graphs via Hierarchical Concepts [12.365451175795338]
  Graph neural networks (GNNs) have led to breakthroughs in domains such as drug discovery, social network analysis, and travel time estimation. They lack interpretability which hinders human trust and thereby deployment to settings with high-stakes decisions. We provide HELP, a novel, inherently interpretable graph pooling approach that reveals how concepts from different GNN layers compose to new ones in later steps.
  arXiv  Detail & Related papers  (2023-11-25T20:06:46Z)
• Explaining Explainability: Towards Deeper Actionable Insights into Deep Learning through Second-order Explainability [70.60433013657693]
  Second-order explainable AI (SOXAI) was recently proposed to extend explainable AI (XAI) from the instance level to the dataset level. We demonstrate for the first time, via example classification and segmentation cases, that eliminating irrelevant concepts from the training set based on actionable insights from SOXAI can enhance a model's performance.
  arXiv  Detail & Related papers  (2023-06-14T23:24:01Z)
• Discriminative Attribution from Counterfactuals [64.94009515033984]
  We present a method for neural network interpretability by combining feature attribution with counterfactual explanations. We show that this method can be used to quantitatively evaluate the performance of feature attribution methods in an objective manner.
  arXiv  Detail & Related papers  (2021-09-28T00:53:34Z)
• CX-ToM: Counterfactual Explanations with Theory-of-Mind for Enhancing Human Trust in Image Recognition Models [84.32751938563426]
  We propose a new explainable AI (XAI) framework for explaining decisions made by a deep convolutional neural network (CNN) In contrast to the current methods in XAI that generate explanations as a single shot response, we pose explanation as an iterative communication process. Our framework generates sequence of explanations in a dialog by mediating the differences between the minds of machine and human user.
  arXiv  Detail & Related papers  (2021-09-03T09:46:20Z)
• A Peek Into the Reasoning of Neural Networks: Interpreting with Structural Visual Concepts [38.215184251799194]
  We propose a framework (VRX) to interpret classification NNs with intuitive structural visual concepts. By means of knowledge distillation, we show VRX can take a step towards mimicking the reasoning process of NNs.
  arXiv  Detail & Related papers  (2021-05-01T15:47:42Z)
• A Minimalist Dataset for Systematic Generalization of Perception, Syntax, and Semantics [131.93113552146195]
  We present a new dataset, Handwritten arithmetic with INTegers (HINT), to examine machines' capability of learning generalizable concepts. In HINT, machines are tasked with learning how concepts are perceived from raw signals such as images. We undertake extensive experiments with various sequence-to-sequence models, including RNNs, Transformers, and GPT-3.
  arXiv  Detail & Related papers  (2021-03-02T01:32:54Z)
• Explainability in Deep Reinforcement Learning [68.8204255655161]
  We review recent works in the direction to attain Explainable Reinforcement Learning (XRL) In critical situations where it is essential to justify and explain the agent's behaviour, better explainability and interpretability of RL models could help gain scientific insight on the inner workings of what is still considered a black box.
  arXiv  Detail & Related papers  (2020-08-15T10:11:42Z)
• Compositional Explanations of Neurons [52.71742655312625]
  We describe a procedure for explaining neurons in deep representations by identifying compositional logical concepts. We use this procedure to answer several questions on interpretability in models for vision and natural language processing.
  arXiv  Detail & Related papers  (2020-06-24T20:37:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.