Towards Human-Interpretable Prototypes for Visual Assessment of Image Classification Models

• URL: http://arxiv.org/abs/2211.12173v1
• Date: Tue, 22 Nov 2022 11:01:22 GMT
• Title: Towards Human-Interpretable Prototypes for Visual Assessment of Image Classification Models
• Authors: Poulami Sinhamahapatra, Lena Heidemann, Maureen Monnet, Karsten Roscher
• Abstract summary: We need models which are interpretable-by-design built on a reasoning process similar to humans. ProtoPNet claims to discover visually meaningful prototypes in an unsupervised way. We find that these prototypes still have a long way ahead towards definite explanations.
• Score: 9.577509224534323
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Explaining black-box Artificial Intelligence (AI) models is a cornerstone for trustworthy AI and a prerequisite for its use in safety critical applications such that AI models can reliably assist humans in critical decisions. However, instead of trying to explain our models post-hoc, we need models which are interpretable-by-design built on a reasoning process similar to humans that exploits meaningful high-level concepts such as shapes, texture or object parts. Learning such concepts is often hindered by its need for explicit specification and annotation up front. Instead, prototype-based learning approaches such as ProtoPNet claim to discover visually meaningful prototypes in an unsupervised way. In this work, we propose a set of properties that those prototypes have to fulfill to enable human analysis, e.g. as part of a reliable model assessment case, and analyse such existing methods in the light of these properties. Given a 'Guess who?' game, we find that these prototypes still have a long way ahead towards definite explanations. We quantitatively validate our findings by conducting a user study indicating that many of the learnt prototypes are not considered useful towards human understanding. We discuss about the missing links in the existing methods and present a potential real-world application motivating the need to progress towards truly human-interpretable prototypes.

Related papers

• Unsupervised Model Diagnosis [49.36194740479798]
  This paper proposes Unsupervised Model Diagnosis (UMO) to produce semantic counterfactual explanations without any user guidance. Our approach identifies and visualizes changes in semantics, and then matches these changes to attributes from wide-ranging text sources.
  arXiv  Detail & Related papers  (2024-10-08T17:59:03Z)
• Enhanced Prototypical Part Network (EPPNet) For Explainable Image Classification Via Prototypes [16.528373143163275]
  We introduce the Enhanced Prototypical Part Network (EPPNet) for image classification. EPPNet achieves strong performance while discovering relevant prototypes that can be used to explain the classification results. Our evaluations on the CUB-200-2011 dataset show that the EPPNet outperforms state-of-the-art xAI-based methods.
  arXiv  Detail & Related papers  (2024-08-08T17:26:56Z)
• Evaluating the Explainability of Attributes and Prototypes for a Medical Classification Model [0.0]
  We evaluate attribute- and prototype-based explanations with the Proto-Caps model. We can conclude that attribute scores and visual prototypes enhance confidence in the model.
  arXiv  Detail & Related papers  (2024-04-15T16:43:24Z)
• Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
  We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development. To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps. These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
  arXiv  Detail & Related papers  (2023-12-10T23:13:23Z)
• ProtoExplorer: Interpretable Forensic Analysis of Deepfake Videos using Prototype Exploration and Refinement [11.182863992851622]
  ProtoExplorer is a Visual Analytics system for exploration and refinement of prototype-based deepfake detection models. It offers tools for visualizing and temporally filtering prototype-based predictions when working with video data. System was designed with forensic experts and evaluated in a number of rounds based on both open-ended think evaluation and interviews.
  arXiv  Detail & Related papers  (2023-09-20T09:03:56Z)
• Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations [97.91630330328815]
  We conduct a crowdsourcing study, where participants interact with deception detection models that have been trained to distinguish between genuine and fake hotel reviews. We observe that for a linear bag-of-words model, participants with access to the feature coefficients during training are able to cause a larger reduction in model confidence in the testing phase when compared to the no-explanation control.
  arXiv  Detail & Related papers  (2021-12-17T18:29:56Z)
• Exploring Alignment of Representations with Human Perception [47.53970721813083]
  We show that inputs that are mapped to similar representations by the model should be perceived similarly by humans. Our approach yields a measure of the extent to which a model is aligned with human perception. We find that various properties of a model like its architecture, training paradigm, training loss, and data augmentation play a significant role in learning representations that are aligned with human perception.
  arXiv  Detail & Related papers  (2021-11-29T17:26:50Z)
• This Looks Like That, Because ... Explaining Prototypes for Interpretable Image Recognition [4.396860522241307]
  We argue that prototypes should be explained. Our method clarifies the meaning of a prototype by quantifying the influence of colour hue, shape, texture, contrast and saturation. By explaining such'misleading' prototypes, we improve the interpretability and simulatability of a prototype-based classification model.
  arXiv  Detail & Related papers  (2020-11-05T14:43:07Z)
• A Diagnostic Study of Explainability Techniques for Text Classification [52.879658637466605]
  We develop a list of diagnostic properties for evaluating existing explainability techniques. We compare the saliency scores assigned by the explainability techniques with human annotations of salient input regions to find relations between a model's performance and the agreement of its rationales with human ones.
  arXiv  Detail & Related papers  (2020-09-25T12:01:53Z)
• Plausible Counterfactuals: Auditing Deep Learning Classifiers with Realistic Adversarial Examples [84.8370546614042]
  Black-box nature of Deep Learning models has posed unanswered questions about what they learn from data. Generative Adversarial Network (GAN) and multi-objectives are used to furnish a plausible attack to the audited model. Its utility is showcased within a human face classification task, unveiling the enormous potential of the proposed framework.
  arXiv  Detail & Related papers  (2020-03-25T11:08:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.