Be Careful When Evaluating Explanations Regarding Ground Truth

• URL: http://arxiv.org/abs/2311.04813v1
• Date: Wed, 8 Nov 2023 16:39:13 GMT
• Title: Be Careful When Evaluating Explanations Regarding Ground Truth
• Authors: Hubert Baniecki, Maciej Chrabaszcz, Andreas Holzinger, Bastian Pfeifer, Anna Saranti, Przemyslaw Biecek
• Abstract summary: evaluating explanations of images regarding ground truth primarily evaluates the quality of the models under consideration rather than the explanation methods themselves. We propose a framework for $textitjointly$ evaluating the discrepancy of systems that align with an explanation system.
• Score: 11.340743580750642
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Evaluating explanations of image classifiers regarding ground truth, e.g. segmentation masks defined by human perception, primarily evaluates the quality of the models under consideration rather than the explanation methods themselves. Driven by this observation, we propose a framework for $\textit{jointly}$ evaluating the robustness of safety-critical systems that $\textit{combine}$ a deep neural network with an explanation method. These are increasingly used in real-world applications like medical image analysis or robotics. We introduce a fine-tuning procedure to (mis)align model$\unicode{x2013}$explanation pipelines with ground truth and use it to quantify the potential discrepancy between worst and best-case scenarios of human alignment. Experiments across various model architectures and post-hoc local interpretation methods provide insights into the robustness of vision transformers and the overall vulnerability of such AI systems to potential adversarial attacks.

Related papers

• Unsupervised Model Diagnosis [49.36194740479798]
  This paper proposes Unsupervised Model Diagnosis (UMO) to produce semantic counterfactual explanations without any user guidance. Our approach identifies and visualizes changes in semantics, and then matches these changes to attributes from wide-ranging text sources.
  arXiv  Detail & Related papers  (2024-10-08T17:59:03Z)
• On the Road to Clarity: Exploring Explainable AI for World Models in a Driver Assistance System [3.13366804259509]
  We build a transparent backbone model for convolutional variational autoencoders (VAE) We propose explanation and evaluation techniques for the internal dynamics and feature relevance of prediction networks. We showcase our methods by analyzing a VAE-LSTM world model that predicts pedestrian perception in an urban traffic situation.
  arXiv  Detail & Related papers  (2024-04-26T11:57:17Z)
• CNN-based explanation ensembling for dataset, representation and explanations evaluation [1.1060425537315088]
  We explore the potential of ensembling explanations generated by deep classification models using convolutional model. Through experimentation and analysis, we aim to investigate the implications of combining explanations to uncover a more coherent and reliable patterns of the model's behavior.
  arXiv  Detail & Related papers  (2024-04-16T08:39:29Z)
• Towards Evaluating Transfer-based Attacks Systematically, Practically, and Fairly [79.07074710460012]
  adversarial vulnerability of deep neural networks (DNNs) has drawn great attention. An increasing number of transfer-based methods have been developed to fool black-box DNN models. We establish a transfer-based attack benchmark (TA-Bench) which implements 30+ methods.
  arXiv  Detail & Related papers  (2023-11-02T15:35:58Z)
• FunnyBirds: A Synthetic Vision Dataset for a Part-Based Analysis of Explainable AI Methods [15.073405675079558]
  XAI inherently lacks ground-truth explanations, making its automatic evaluation an unsolved problem. We propose a novel synthetic vision dataset, named FunnyBirds, and accompanying automatic evaluation protocols. Using our tools, we report results for 24 different combinations of neural models and XAI methods.
  arXiv  Detail & Related papers  (2023-08-11T17:29:02Z)
• Explanation Method for Anomaly Detection on Mixed Numerical and Categorical Spaces [0.9543943371833464]
  We present EADMNC (Explainable Anomaly Detection on Mixed Numerical and Categorical spaces) It adds explainability to the predictions obtained with the original model. We report experimental results on extensive real-world data, particularly in the domain of network intrusion detection.
  arXiv  Detail & Related papers  (2022-09-09T08:20:13Z)
• Explainers in the Wild: Making Surrogate Explainers Robust to Distortions through Perception [77.34726150561087]
  We propose a methodology to evaluate the effect of distortions in explanations by embedding perceptual distances. We generate explanations for images in the Imagenet-C dataset and demonstrate how using a perceptual distances in the surrogate explainer creates more coherent explanations for the distorted and reference images.
  arXiv  Detail & Related papers  (2021-02-22T12:38:53Z)
• Deep Co-Attention Network for Multi-View Subspace Learning [73.3450258002607]
  We propose a deep co-attention network for multi-view subspace learning. It aims to extract both the common information and the complementary information in an adversarial setting. In particular, it uses a novel cross reconstruction loss and leverages the label information to guide the construction of the latent representation.
  arXiv  Detail & Related papers  (2021-02-15T18:46:44Z)
• This is not the Texture you are looking for! Introducing Novel Counterfactual Explanations for Non-Experts using Generative Adversarial Learning [59.17685450892182]
  counterfactual explanation systems try to enable a counterfactual reasoning by modifying the input image. We present a novel approach to generate such counterfactual image explanations based on adversarial image-to-image translation techniques. Our results show that our approach leads to significantly better results regarding mental models, explanation satisfaction, trust, emotions, and self-efficacy than two state-of-the art systems.
  arXiv  Detail & Related papers  (2020-12-22T10:08:05Z)
• Ground Truth Evaluation of Neural Network Explanations with CLEVR-XAI [12.680653816836541]
  We propose a ground truth based evaluation framework for XAI methods based on the CLEVR visual question answering task. Our framework provides a (1) selective, (2) controlled and (3) realistic testbed for the evaluation of neural network explanations.
  arXiv  Detail & Related papers  (2020-03-16T14:43:33Z)
• Explainable Deep Classification Models for Domain Generalization [94.43131722655617]
  Explanations are defined as regions of visual evidence upon which a deep classification network makes a decision. Our training strategy enforces a periodic saliency-based feedback to encourage the model to focus on the image regions that directly correspond to the ground-truth object.
  arXiv  Detail & Related papers  (2020-03-13T22:22:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.