Interpretable Image Classification via Non-parametric Part Prototype Learning

• URL: http://arxiv.org/abs/2503.10247v1
• Date: Thu, 13 Mar 2025 10:46:53 GMT
• Title: Interpretable Image Classification via Non-parametric Part Prototype Learning
• Authors: Zhijie Zhu, Lei Fan, Maurice Pagnucco, Yang Song,
• Abstract summary: Classifying images with an interpretable decision-making process is a long-standing problem in computer vision.<n>In recent years, Prototypical Part Networks has gained traction as an approach for self-explainable neural networks.<n>We present a framework for part-based interpretable image classification that learns a set of semantically distinctive object parts for each class.
• Score: 14.390730075612248
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Classifying images with an interpretable decision-making process is a long-standing problem in computer vision. In recent years, Prototypical Part Networks has gained traction as an approach for self-explainable neural networks, due to their ability to mimic human visual reasoning by providing explanations based on prototypical object parts. However, the quality of the explanations generated by these methods leaves room for improvement, as the prototypes usually focus on repetitive and redundant concepts. Leveraging recent advances in prototype learning, we present a framework for part-based interpretable image classification that learns a set of semantically distinctive object parts for each class, and provides diverse and comprehensive explanations. The core of our method is to learn the part-prototypes in a non-parametric fashion, through clustering deep features extracted from foundation vision models that encode robust semantic information. To quantitatively evaluate the quality of explanations provided by ProtoPNets, we introduce Distinctiveness Score and Comprehensiveness Score. Through evaluation on CUB-200-2011, Stanford Cars and Stanford Dogs datasets, we show that our framework compares favourably against existing ProtoPNets while achieving better interpretability. Code is available at: https://github.com/zijizhu/proto-non-param.

Related papers

• From Visual Explanations to Counterfactual Explanations with Latent Diffusion [11.433402357922414]
  We propose a new approach to tackle two key challenges in recent prominent works. First, we determine which specific counterfactual features are crucial for distinguishing the "concept" of the target class from the original class. Second, we provide valuable explanations for the non-robust classifier without relying on the support of an adversarially robust model.
  arXiv  Detail & Related papers  (2025-04-12T13:04:00Z)
• COMIX: Compositional Explanations using Prototypes [46.15031477955461]
  We propose a method to align machine representations with human understanding.<n>The proposed method, named COMIX, classifies an image by decomposing it into regions based on learned concepts.<n>We show that our method provides fidelity of explanations and shows that the efficiency is competitive with other inherently interpretable architectures.
  arXiv  Detail & Related papers  (2025-01-10T15:40:31Z)
• Enhanced Prototypical Part Network (EPPNet) For Explainable Image Classification Via Prototypes [16.528373143163275]
  We introduce the Enhanced Prototypical Part Network (EPPNet) for image classification. EPPNet achieves strong performance while discovering relevant prototypes that can be used to explain the classification results. Our evaluations on the CUB-200-2011 dataset show that the EPPNet outperforms state-of-the-art xAI-based methods.
  arXiv  Detail & Related papers  (2024-08-08T17:26:56Z)
• ProtoS-ViT: Visual foundation models for sparse self-explainable classifications [0.6249768559720122]
  Prototypical networks aim to build intrinsically explainable models based on the linear summation of concepts.<n>This work first proposes an extensive set of quantitative and qualitative metrics which allow to identify drawbacks in current prototypical networks.<n>It then introduces a novel architecture which provides compact explanations, outperforming current prototypical models in terms of explanation quality.
  arXiv  Detail & Related papers  (2024-06-14T13:36:30Z)
• MCPNet: An Interpretable Classifier via Multi-Level Concept Prototypes [24.28807025839685]
  We argue that explanations lacking insights into the decision processes of low and mid-level features are neither fully faithful nor useful. We propose a novel paradigm that learns and aligns multi-level concept prototype distributions for classification purposes via Class-aware Concept Distribution (CCD) loss.
  arXiv  Detail & Related papers  (2024-04-13T11:13:56Z)
• Neural Clustering based Visual Representation Learning [61.72646814537163]
  Clustering is one of the most classic approaches in machine learning and data analysis. We propose feature extraction with clustering (FEC), which views feature extraction as a process of selecting representatives from data. FEC alternates between grouping pixels into individual clusters to abstract representatives and updating the deep features of pixels with current representatives.
  arXiv  Detail & Related papers  (2024-03-26T06:04:50Z)
• Unsupervised Part Discovery from Contrastive Reconstruction [90.88501867321573]
  The goal of self-supervised visual representation learning is to learn strong, transferable image representations. We propose an unsupervised approach to object part discovery and segmentation. Our method yields semantic parts consistent across fine-grained but visually distinct categories.
  arXiv  Detail & Related papers  (2021-11-11T17:59:42Z)
• A Diagnostic Study of Explainability Techniques for Text Classification [52.879658637466605]
  We develop a list of diagnostic properties for evaluating existing explainability techniques. We compare the saliency scores assigned by the explainability techniques with human annotations of salient input regions to find relations between a model's performance and the agreement of its rationales with human ones.
  arXiv  Detail & Related papers  (2020-09-25T12:01:53Z)
• Region Comparison Network for Interpretable Few-shot Image Classification [97.97902360117368]
  Few-shot image classification has been proposed to effectively use only a limited number of labeled examples to train models for new classes. We propose a metric learning based method named Region Comparison Network (RCN), which is able to reveal how few-shot learning works. We also present a new way to generalize the interpretability from the level of tasks to categories.
  arXiv  Detail & Related papers  (2020-09-08T07:29:05Z)
• Part-aware Prototype Network for Few-shot Semantic Segmentation [50.581647306020095]
  We propose a novel few-shot semantic segmentation framework based on the prototype representation. Our key idea is to decompose the holistic class representation into a set of part-aware prototypes. We develop a novel graph neural network model to generate and enhance the proposed part-aware prototypes.
  arXiv  Detail & Related papers  (2020-07-13T11:03:09Z)
• Prototypical Contrastive Learning of Unsupervised Representations [171.3046900127166]
  Prototypical Contrastive Learning (PCL) is an unsupervised representation learning method. PCL implicitly encodes semantic structures of the data into the learned embedding space. PCL outperforms state-of-the-art instance-wise contrastive learning methods on multiple benchmarks.
  arXiv  Detail & Related papers  (2020-05-11T09:53:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.