Explaining and visualizing black-box models through counterfactual paths

• URL: http://arxiv.org/abs/2307.07764v3
• Date: Tue, 1 Aug 2023 07:01:31 GMT
• Title: Explaining and visualizing black-box models through counterfactual paths
• Authors: Bastian Pfeifer, Mateusz Krzyzinski, Hubert Baniecki, Anna Saranti, Andreas Holzinger, Przemyslaw Biecek
• Abstract summary: We propose a novel approach to explainable AI (XAI) that uses the so-called counterfactual paths generated by conditional permutations of features. The algorithm measures feature importance by identifying sequential permutations of features that most influence changes in model predictions. It is particularly suitable for generating explanations based on counterfactual paths in knowledge graphs incorporating domain knowledge.
• Score: 5.930734371401315
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Explainable AI (XAI) is an increasingly important area of machine learning research, which aims to make black-box models transparent and interpretable. In this paper, we propose a novel approach to XAI that uses the so-called counterfactual paths generated by conditional permutations of features. The algorithm measures feature importance by identifying sequential permutations of features that most influence changes in model predictions. It is particularly suitable for generating explanations based on counterfactual paths in knowledge graphs incorporating domain knowledge. Counterfactual paths introduce an additional graph dimension to current XAI methods in both explaining and visualizing black-box models. Experiments with synthetic and medical data demonstrate the practical applicability of our approach.

Related papers

• From Latent to Lucid: Transforming Knowledge Graph Embeddings into Interpretable Structures [2.6451388057494283]
  This paper introduces a post-hoc explainable AI method tailored for Knowledge Graph Embedding models. Our approach directly decodes the latent representations encoded by Knowledge Graph Embedding models. By identifying distinct structures within the subgraph neighborhoods of similarly embedded entities, our method translates these insights into human-understandable symbolic rules and facts.
  arXiv  Detail & Related papers  (2024-06-03T19:54:11Z)
• Enhancing Counterfactual Image Generation Using Mahalanobis Distance with Distribution Preferences in Feature Space [7.00851481261778]
  In the realm of Artificial Intelligence (AI), the importance of Explainable Artificial Intelligence (XAI) is increasingly recognized. One notable single-instance XAI approach is counterfactual explanation, which aids users in comprehending a model's decisions. This paper introduces a novel method for computing feature importance within the feature space of a black-box model.
  arXiv  Detail & Related papers  (2024-05-31T08:26:53Z)
• Interaction as Explanation: A User Interaction-based Method for Explaining Image Classification Models [1.3597551064547502]
  In computer vision, explainable AI (xAI) methods seek to mitigate the 'black-box' problem. Traditional xAI methods concentrate on visualizing input features that influence model predictions. We present an interaction-based xAI method that enhances user comprehension of image classification models through their interaction.
  arXiv  Detail & Related papers  (2024-04-15T14:26:00Z)
• Explanations of Black-Box Models based on Directional Feature Interactions [8.25114410474287]
  We show how to explain black-box models by capturing feature interactions in a directed graph. We show the superiority of our method against state-of-the-art on IMDB10, Census, Divorce, Drug, and gene data.
  arXiv  Detail & Related papers  (2023-04-16T02:00:25Z)
• Visualizing Transferred Knowledge: An Interpretive Model of Unsupervised Domain Adaptation [70.85686267987744]
  Unsupervised domain adaptation problems can transfer knowledge from a labeled source domain to an unlabeled target domain. We propose an interpretive model of unsupervised domain adaptation, as the first attempt to visually unveil the mystery of transferred knowledge. Our method provides an intuitive explanation for the base model's predictions and unveils transfer knowledge by matching the image patches with the same semantics across both source and target domains.
  arXiv  Detail & Related papers  (2023-03-04T03:02:12Z)
• MACE: An Efficient Model-Agnostic Framework for Counterfactual Explanation [132.77005365032468]
  We propose a novel framework of Model-Agnostic Counterfactual Explanation (MACE) In our MACE approach, we propose a novel RL-based method for finding good counterfactual examples and a gradient-less descent method for improving proximity. Experiments on public datasets validate the effectiveness with better validity, sparsity and proximity.
  arXiv  Detail & Related papers  (2022-05-31T04:57:06Z)
• Quasi Black-Box Variational Inference with Natural Gradients for Bayesian Learning [84.90242084523565]
  We develop an optimization algorithm suitable for Bayesian learning in complex models. Our approach relies on natural gradient updates within a general black-box framework for efficient training with limited model-specific derivations.
  arXiv  Detail & Related papers  (2022-05-23T18:54:27Z)
• Beyond Explaining: Opportunities and Challenges of XAI-Based Model Improvement [75.00655434905417]
  Explainable Artificial Intelligence (XAI) is an emerging research field bringing transparency to highly complex machine learning (ML) models. This paper offers a comprehensive overview over techniques that apply XAI practically for improving various properties of ML models. We show empirically through experiments on toy and realistic settings how explanations can help improve properties such as model generalization ability or reasoning.
  arXiv  Detail & Related papers  (2022-03-15T15:44:28Z)
• Towards a Shapley Value Graph Framework for Medical peer-influence [0.9449650062296824]
  This paper introduces a new framework to look deeper into explanations using graph representation for feature-to-feature interactions. It aims to improve the interpretability of black-box Machine Learning (ML) models and inform intervention.
  arXiv  Detail & Related papers  (2021-12-29T16:24:50Z)
• Bayesian Graph Contrastive Learning [55.36652660268726]
  We propose a novel perspective of graph contrastive learning methods showing random augmentations leads to encoders. Our proposed method represents each node by a distribution in the latent space in contrast to existing techniques which embed each node to a deterministic vector. We show a considerable improvement in performance compared to existing state-of-the-art methods on several benchmark datasets.
  arXiv  Detail & Related papers  (2021-12-15T01:45:32Z)
• Learning outside the Black-Box: The pursuit of interpretable models [78.32475359554395]
  This paper proposes an algorithm that produces a continuous global interpretation of any given continuous black-box function. Our interpretation represents a leap forward from the previous state of the art.
  arXiv  Detail & Related papers  (2020-11-17T12:39:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.