Sanity checks for patch visualisation in prototype-based image classification

• URL: http://arxiv.org/abs/2311.16120v1
• Date: Wed, 25 Oct 2023 08:13:02 GMT
• Title: Sanity checks for patch visualisation in prototype-based image classification
• Authors: Romain Xu-Darme (LSL, LIG), Georges Qu\'enot (LIG), Zakaria Chihani (LSL), Marie-Christine Rousset (LIG)
• Abstract summary: We show that the visualisation methods implemented in ProtoPNet and ProtoTree do not correctly identify the regions of interest inside of the images. We also demonstrate quantitatively that this issue can be mitigated by using other saliency methods that provide more faithful image patches.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work, we perform an analysis of the visualisation methods implemented in ProtoPNet and ProtoTree, two self-explaining visual classifiers based on prototypes. We show that such methods do not correctly identify the regions of interest inside of the images, and therefore do not reflect the model behaviour, which can create a false sense of bias in the model. We also demonstrate quantitatively that this issue can be mitigated by using other saliency methods that provide more faithful image patches.

Related papers

• Hybrid diffusion models: combining supervised and generative pretraining for label-efficient fine-tuning of segmentation models [55.2480439325792]
  We propose a new pretext task, which is to perform simultaneously image denoising and mask prediction on the first domain. We show that fine-tuning a model pretrained using this approach leads to better results than fine-tuning a similar model trained using either supervised or unsupervised pretraining.
  arXiv  Detail & Related papers  (2024-08-06T20:19:06Z)
• ProtoP-OD: Explainable Object Detection with Prototypical Parts [0.0]
  This paper introduces an extension to detection transformers that constructs prototypical local features and uses them in object detection. The proposed extension consists of a bottleneck module, the prototype neck, that computes a discretized representation of prototype activations.
  arXiv  Detail & Related papers  (2024-02-29T13:25:15Z)
• This Looks Like Those: Illuminating Prototypical Concepts Using Multiple Visualizations [19.724372592639774]
  ProtoConcepts is a method for interpretable image classification combining deep learning and case-based reasoning. Our proposed method modifies the architecture of prototype-based networks to instead learn concepts which are visualized using multiple image patches. Our experiments show that our this looks like those'' reasoning process can be applied as a modification to a wide range of existing prototypical image classification networks.
  arXiv  Detail & Related papers  (2023-10-28T04:54:48Z)
• Zero-shot Model Diagnosis [80.36063332820568]
  A common approach to evaluate deep learning models is to build a labeled test set with attributes of interest and assess how well it performs. This paper argues the case that Zero-shot Model Diagnosis (ZOOM) is possible without the need for a test set nor labeling.
  arXiv  Detail & Related papers  (2023-03-27T17:59:33Z)
• Sanity checks and improvements for patch visualisation in prototype-based image classification [0.0]
  We perform an in-depth analysis of the visualisation methods implemented in two popular self-explaining models for visual classification based on prototypes. We first show that such methods do not correctly identify the regions of interest inside of the images, and therefore do not reflect the model behaviour. We discuss the implications of our findings for other prototype-based models sharing the same visualisation method.
  arXiv  Detail & Related papers  (2023-01-20T15:13:04Z)
• DisPositioNet: Disentangled Pose and Identity in Semantic Image Manipulation [83.51882381294357]
  DisPositioNet is a model that learns a disentangled representation for each object for the task of image manipulation using scene graphs. Our framework enables the disentanglement of the variational latent embeddings as well as the feature representation in the graph.
  arXiv  Detail & Related papers  (2022-11-10T11:47:37Z)
• Instance Localization for Self-supervised Detection Pretraining [68.24102560821623]
  We propose a new self-supervised pretext task, called instance localization. We show that integration of bounding boxes into pretraining promotes better task alignment and architecture alignment for transfer learning. Experimental results demonstrate that our approach yields state-of-the-art transfer learning results for object detection.
  arXiv  Detail & Related papers  (2021-02-16T17:58:57Z)
• This Looks Like That, Because ... Explaining Prototypes for Interpretable Image Recognition [4.396860522241307]
  We argue that prototypes should be explained. Our method clarifies the meaning of a prototype by quantifying the influence of colour hue, shape, texture, contrast and saturation. By explaining such'misleading' prototypes, we improve the interpretability and simulatability of a prototype-based classification model.
  arXiv  Detail & Related papers  (2020-11-05T14:43:07Z)
• Synthesizing the Unseen for Zero-shot Object Detection [72.38031440014463]
  We propose to synthesize visual features for unseen classes, so that the model learns both seen and unseen objects in the visual domain. We use a novel generative model that uses class-semantics to not only generate the features but also to discriminatively separate them.
  arXiv  Detail & Related papers  (2020-10-19T12:36:11Z)
• Towards Visually Explaining Similarity Models [29.704524987493766]
  We present a method to generate gradient-based visual attention for image similarity predictors. By relying solely on the learned feature embedding, we show that our approach can be applied to any kind of CNN-based similarity architecture. We show that our resulting attention maps serve more than just interpretability; they can be infused into the model learning process itself with new trainable constraints.
  arXiv  Detail & Related papers  (2020-08-13T17:47:41Z)
• Distilling Localization for Self-Supervised Representation Learning [82.79808902674282]
  Contrastive learning has revolutionized unsupervised representation learning. Current contrastive models are ineffective at localizing the foreground object. We propose a data-driven approach for learning in variance to backgrounds.
  arXiv  Detail & Related papers  (2020-04-14T16:29:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.