Expressive Explanations of DNNs by Combining Concept Analysis with ILP
- URL: http://arxiv.org/abs/2105.07371v1
- Date: Sun, 16 May 2021 07:00:27 GMT
- Title: Expressive Explanations of DNNs by Combining Concept Analysis with ILP
- Authors: Johannes Rabold, Gesina Schwalbe, Ute Schmid
- Abstract summary: We use inherent features learned by the network to build a global, expressive, verbal explanation of the rationale of a feed-forward convolutional deep neural network (DNN)
We show that our explanation is faithful to the original black-box model.
- Score: 0.3867363075280543
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Explainable AI has emerged to be a key component for black-box machine
learning approaches in domains with a high demand for reliability or
transparency. Examples are medical assistant systems, and applications
concerned with the General Data Protection Regulation of the European Union,
which features transparency as a cornerstone. Such demands require the ability
to audit the rationale behind a classifier's decision. While visualizations are
the de facto standard of explanations, they come short in terms of
expressiveness in many ways: They cannot distinguish between different
attribute manifestations of visual features (e.g. eye open vs. closed), and
they cannot accurately describe the influence of absence of, and relations
between features. An alternative would be more expressive symbolic surrogate
models. However, these require symbolic inputs, which are not readily available
in most computer vision tasks. In this paper we investigate how to overcome
this: We use inherent features learned by the network to build a global,
expressive, verbal explanation of the rationale of a feed-forward convolutional
deep neural network (DNN). The semantics of the features are mined by a concept
analysis approach trained on a set of human understandable visual concepts. The
explanation is found by an Inductive Logic Programming (ILP) method and
presented as first-order rules. We show that our explanation is faithful to the
original black-box model.
The code for our experiments is available at
https://github.com/mc-lovin-mlem/concept-embeddings-and-ilp/tree/ki2020.
Related papers
- Help Me Identify: Is an LLM+VQA System All We Need to Identify Visual Concepts? [62.984473889987605]
We present a zero-shot framework for fine-grained visual concept learning by leveraging large language model and Visual Question Answering (VQA) system.
We pose these questions along with the query image to a VQA system and aggregate the answers to determine the presence or absence of an object in the test images.
Our experiments demonstrate comparable performance with existing zero-shot visual classification methods and few-shot concept learning approaches.
arXiv Detail & Related papers (2024-10-17T15:16:10Z) - Relevant Irrelevance: Generating Alterfactual Explanations for Image Classifiers [11.200613814162185]
In this paper, we demonstrate the feasibility of alterfactual explanations for black box image classifiers.
We show for the first time that it is possible to apply this idea to black box models based on neural networks.
arXiv Detail & Related papers (2024-05-08T11:03:22Z) - Identifying Interpretable Subspaces in Image Representations [54.821222487956355]
We propose a framework to explain features of image representations using Contrasting Concepts (FALCON)
For a target feature, FALCON captions its highly activating cropped images using a large captioning dataset and a pre-trained vision-language model like CLIP.
Each word among the captions is scored and ranked leading to a small number of shared, human-understandable concepts.
arXiv Detail & Related papers (2023-07-20T00:02:24Z) - Dynamic Clue Bottlenecks: Towards Interpretable-by-Design Visual Question Answering [58.64831511644917]
We introduce an interpretable by design model that factors model decisions into intermediate human-legible explanations.
We show that our inherently interpretable system can improve 4.64% over a comparable black-box system in reasoning-focused questions.
arXiv Detail & Related papers (2023-05-24T08:33:15Z) - TCNL: Transparent and Controllable Network Learning Via Embedding
Human-Guided Concepts [10.890006696574803]
We propose a novel method, Transparent and Controllable Network Learning (TCNL), to overcome such challenges.
Towards the goal of improving transparency-interpretability, in TCNL, we define some concepts for specific classification tasks through scientific human-intuition study.
We also build the concept mapper to visualize features extracted by the concept extractor in a human-intuitive way.
arXiv Detail & Related papers (2022-10-07T01:18:37Z) - Emergence of Machine Language: Towards Symbolic Intelligence with Neural
Networks [73.94290462239061]
We propose to combine symbolism and connectionism principles by using neural networks to derive a discrete representation.
By designing an interactive environment and task, we demonstrated that machines could generate a spontaneous, flexible, and semantic language.
arXiv Detail & Related papers (2022-01-14T14:54:58Z) - Logic Explained Networks [27.800583434727805]
We show how a mindful design of the networks leads to a family of interpretable deep learning models called Logic Explained Networks (LENs)
LENs only require their inputs to be human-understandable predicates, and they provide explanations in terms of simple First-Order Logic (FOL) formulas.
LENs may yield better classifications than established white-box models, such as decision trees and Bayesian rule lists.
arXiv Detail & Related papers (2021-08-11T10:55:42Z) - Contrastive Explanations for Model Interpretability [77.92370750072831]
We propose a methodology to produce contrastive explanations for classification models.
Our method is based on projecting model representation to a latent space.
Our findings shed light on the ability of label-contrastive explanations to provide a more accurate and finer-grained interpretability of a model's decision.
arXiv Detail & Related papers (2021-03-02T00:36:45Z) - This is not the Texture you are looking for! Introducing Novel
Counterfactual Explanations for Non-Experts using Generative Adversarial
Learning [59.17685450892182]
counterfactual explanation systems try to enable a counterfactual reasoning by modifying the input image.
We present a novel approach to generate such counterfactual image explanations based on adversarial image-to-image translation techniques.
Our results show that our approach leads to significantly better results regarding mental models, explanation satisfaction, trust, emotions, and self-efficacy than two state-of-the art systems.
arXiv Detail & Related papers (2020-12-22T10:08:05Z) - Explanations of Black-Box Model Predictions by Contextual Importance and
Utility [1.7188280334580195]
We present the Contextual Importance (CI) and Contextual Utility (CU) concepts to extract explanations easily understandable by experts as well as novice users.
This method explains the prediction results without transforming the model into an interpretable one.
We show the utility of explanations in car selection example and Iris flower classification by presenting complete (i.e. the causes of an individual prediction) and contrastive explanation.
arXiv Detail & Related papers (2020-05-30T06:49:50Z) - LIMEtree: Interactively Customisable Explanations Based on Local
Surrogate Multi-output Regression Trees [21.58324172085553]
We introduce a model-agnostic and post-hoc local explainability technique for black-box predictions called LIMEtree.
We validate our algorithm on a deep neural network trained for object detection in images and compare it against Local Interpretable Model-agnostic Explanations (LIME)
Our method comes with local fidelity guarantees and can produce a range of diverse explanation types.
arXiv Detail & Related papers (2020-05-04T12:31:29Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.