Explanations of Black-Box Models based on Directional Feature Interactions

• URL: http://arxiv.org/abs/2304.07670v1
• Date: Sun, 16 Apr 2023 02:00:25 GMT
• Title: Explanations of Black-Box Models based on Directional Feature Interactions
• Authors: Aria Masoomi, Davin Hill, Zhonghui Xu, Craig P Hersh, Edwin K. Silverman, Peter J. Castaldi, Stratis Ioannidis, Jennifer Dy
• Abstract summary: We show how to explain black-box models by capturing feature interactions in a directed graph. We show the superiority of our method against state-of-the-art on IMDB10, Census, Divorce, Drug, and gene data.
• Score: 8.25114410474287
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As machine learning algorithms are deployed ubiquitously to a variety of domains, it is imperative to make these often black-box models transparent. Several recent works explain black-box models by capturing the most influential features for prediction per instance; such explanation methods are univariate, as they characterize importance per feature. We extend univariate explanation to a higher-order; this enhances explainability, as bivariate methods can capture feature interactions in black-box models, represented as a directed graph. Analyzing this graph enables us to discover groups of features that are equally important (i.e., interchangeable), while the notion of directionality allows us to identify the most influential features. We apply our bivariate method on Shapley value explanations, and experimentally demonstrate the ability of directional explanations to discover feature interactions. We show the superiority of our method against state-of-the-art on CIFAR10, IMDB, Census, Divorce, Drug, and gene data.

Related papers

• MOUNTAINEER: Topology-Driven Visual Analytics for Comparing Local Explanations [6.835413642522898]
  Topological Data Analysis (TDA) can be an effective method in this domain since it can be used to transform attributions into uniform graph representations. We present a novel topology-driven visual analytics tool, Mountaineer, that allows ML practitioners to interactively analyze and compare these representations. We show how Mountaineer enabled us to compare black-box ML explanations and discern regions of and causes of disagreements between different explanations.
  arXiv  Detail & Related papers  (2024-06-21T19:28:50Z)
• A Unified Framework for Input Feature Attribution Analysis [44.5176896474762]
  One popular explanation form highlights key input features, such as tokens and interactions between tokens. These explanation types have only been studied in isolation, making it difficult to judge their respective applicability. In our experiments, highlight explanations are the most faithful to a model's prediction, and interactive explanations provide better utility for learning to simulate a model's predictions.
  arXiv  Detail & Related papers  (2024-06-21T12:01:03Z)
• Diffexplainer: Towards Cross-modal Global Explanations with Diffusion Models [51.21351775178525]
  DiffExplainer is a novel framework that, leveraging language-vision models, enables multimodal global explainability. It employs diffusion models conditioned on optimized text prompts, synthesizing images that maximize class outputs. The analysis of generated visual descriptions allows for automatic identification of biases and spurious features.
  arXiv  Detail & Related papers  (2024-04-03T10:11:22Z)
• Visual Commonsense based Heterogeneous Graph Contrastive Learning [79.22206720896664]
  We propose a heterogeneous graph contrastive learning method to better finish the visual reasoning task. Our method is designed as a plug-and-play way, so that it can be quickly and easily combined with a wide range of representative methods.
  arXiv  Detail & Related papers  (2023-11-11T12:01:18Z)
• Explaining and visualizing black-box models through counterfactual paths [5.930734371401315]
  We propose a novel approach to explainable AI (XAI) that uses the so-called counterfactual paths generated by conditional permutations of features. The algorithm measures feature importance by identifying sequential permutations of features that most influence changes in model predictions. It is particularly suitable for generating explanations based on counterfactual paths in knowledge graphs incorporating domain knowledge.
  arXiv  Detail & Related papers  (2023-07-15T10:16:51Z)
• Learning with Explanation Constraints [91.23736536228485]
  We provide a learning theoretic framework to analyze how explanations can improve the learning of our models. We demonstrate the benefits of our approach over a large array of synthetic and real-world experiments.
  arXiv  Detail & Related papers  (2023-03-25T15:06:47Z)
• GRAPHSHAP: Explaining Identity-Aware Graph Classifiers Through the Language of Motifs [11.453325862543094]
  GRAPHSHAP is able to provide motif-based explanations for identity-aware graph classifiers. We show how a simple kernel can efficiently approximate explanation scores, thus allowing GRAPHSHAP to scale on scenarios with a large explanation space. Our experiments highlight how the classification provided by a black-box model can be effectively explained by few connectomics patterns.
  arXiv  Detail & Related papers  (2022-02-17T18:29:30Z)
• Discriminative Attribution from Counterfactuals [64.94009515033984]
  We present a method for neural network interpretability by combining feature attribution with counterfactual explanations. We show that this method can be used to quantitatively evaluate the performance of feature attribution methods in an objective manner.
  arXiv  Detail & Related papers  (2021-09-28T00:53:34Z)
• Contrastive Explanations for Model Interpretability [77.92370750072831]
  We propose a methodology to produce contrastive explanations for classification models. Our method is based on projecting model representation to a latent space. Our findings shed light on the ability of label-contrastive explanations to provide a more accurate and finer-grained interpretability of a model's decision.
  arXiv  Detail & Related papers  (2021-03-02T00:36:45Z)
• Learning outside the Black-Box: The pursuit of interpretable models [78.32475359554395]
  This paper proposes an algorithm that produces a continuous global interpretation of any given continuous black-box function. Our interpretation represents a leap forward from the previous state of the art.
  arXiv  Detail & Related papers  (2020-11-17T12:39:44Z)
• High Dimensional Model Explanations: an Axiomatic Approach [14.908684655206494]
  Complex black-box machine learning models are regularly used in critical decision-making domains. We propose a novel high dimension model explanation method that captures the joint effect of feature subsets.
  arXiv  Detail & Related papers  (2020-06-16T07:48:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.