Mixture of Gaussian-distributed Prototypes with Generative Modelling for Interpretable and Trustworthy Image Recognition

• URL: http://arxiv.org/abs/2312.00092v2
• Date: Wed, 5 Jun 2024 17:03:02 GMT
• Title: Mixture of Gaussian-distributed Prototypes with Generative Modelling for Interpretable and Trustworthy Image Recognition
• Authors: Chong Wang, Yuanhong Chen, Fengbei Liu, Yuyuan Liu, Davis James McCarthy, Helen Frazer, Gustavo Carneiro,
• Abstract summary: We present a new generative paradigm to learn prototype distributions, termed as Mixture of Gaussian-distributed Prototypes (MGProto) MGProto achieves state-of-the-art image recognition and OoD detection performances, while providing encouraging interpretability results.
• Score: 15.685927265270085
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Prototypical-part methods, e.g., ProtoPNet, enhance interpretability in image recognition by linking predictions to training prototypes, thereby offering intuitive insights into their decision-making. Existing methods, which rely on a point-based learning of prototypes, typically face two critical issues: 1) the learned prototypes have limited representation power and are not suitable to detect Out-of-Distribution (OoD) inputs, reducing their decision trustworthiness; and 2) the necessary projection of the learned prototypes back into the space of training images causes a drastic degradation in the predictive performance. Furthermore, current prototype learning adopts an aggressive approach that considers only the most active object parts during training, while overlooking sub-salient object regions which still hold crucial classification information. In this paper, we present a new generative paradigm to learn prototype distributions, termed as Mixture of Gaussian-distributed Prototypes (MGProto). The distribution of prototypes from MGProto enables both interpretable image classification and trustworthy recognition of OoD inputs. The optimisation of MGProto naturally projects the learned prototype distributions back into the training image space, thereby addressing the performance degradation caused by prototype projection. Additionally, we develop a novel and effective prototype mining strategy that considers not only the most active but also sub-salient object parts. To promote model compactness, we further propose to prune MGProto by removing prototypes with low importance priors. Experiments on CUB-200-2011, Stanford Cars, Stanford Dogs, and Oxford-IIIT Pets datasets show that MGProto achieves state-of-the-art image recognition and OoD detection performances, while providing encouraging interpretability results.

Related papers

• Mixed Prototype Consistency Learning for Semi-supervised Medical Image Segmentation [0.0]
  We propose the Mixed Prototype Consistency Learning (MPCL) framework, which includes a Mean Teacher and an auxiliary network. The Mean Teacher generates prototypes for labeled and unlabeled data, while the auxiliary network produces additional prototypes for mixed data processed by CutMix. High-quality global prototypes for each class are formed by fusing two enhanced prototypes, optimizing the distribution of hidden embeddings used in consistency learning.
  arXiv  Detail & Related papers  (2024-04-16T16:51:12Z)
• Query-guided Prototype Evolution Network for Few-Shot Segmentation [85.75516116674771]
  We present a new method that integrates query features into the generation process of foreground and background prototypes. Experimental results on the PASCAL-$5i$ and mirroring-$20i$ datasets attest to the substantial enhancements achieved by QPENet.
  arXiv  Detail & Related papers  (2024-03-11T07:50:40Z)
• ProtoP-OD: Explainable Object Detection with Prototypical Parts [0.0]
  This paper introduces an extension to detection transformers that constructs prototypical local features and uses them in object detection. The proposed extension consists of a bottleneck module, the prototype neck, that computes a discretized representation of prototype activations.
  arXiv  Detail & Related papers  (2024-02-29T13:25:15Z)
• ProtoDiff: Learning to Learn Prototypical Networks by Task-Guided Diffusion [44.805452233966534]
  Prototype-based meta-learning has emerged as a powerful technique for addressing few-shot learning challenges. We introduce ProtoDiff, a framework that gradually generates task-specific prototypes from random noise. We conduct thorough ablation studies to demonstrate its ability to accurately capture the underlying prototype distribution.
  arXiv  Detail & Related papers  (2023-06-26T15:26:24Z)
• Automatically Discovering Novel Visual Categories with Self-supervised Prototype Learning [68.63910949916209]
  This paper tackles the problem of novel category discovery (NCD), which aims to discriminate unknown categories in large-scale image collections. We propose a novel adaptive prototype learning method consisting of two main stages: prototypical representation learning and prototypical self-training. We conduct extensive experiments on four benchmark datasets and demonstrate the effectiveness and robustness of the proposed method with state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-08-01T16:34:33Z)
• Dual Prototypical Contrastive Learning for Few-shot Semantic Segmentation [55.339405417090084]
  We propose a dual prototypical contrastive learning approach tailored to the few-shot semantic segmentation (FSS) task. The main idea is to encourage the prototypes more discriminative by increasing inter-class distance while reducing intra-class distance in prototype feature space. We demonstrate that the proposed dual contrastive learning approach outperforms state-of-the-art FSS methods on PASCAL-5i and COCO-20i datasets.
  arXiv  Detail & Related papers  (2021-11-09T08:14:50Z)
• Prototype Completion for Few-Shot Learning [13.63424509914303]
  Few-shot learning aims to recognize novel classes with few examples. Pre-training based methods effectively tackle the problem by pre-training a feature extractor and then fine-tuning it through the nearest centroid based meta-learning. We propose a novel prototype completion based meta-learning framework.
  arXiv  Detail & Related papers  (2021-08-11T03:44:00Z)
• Attentional Prototype Inference for Few-Shot Segmentation [128.45753577331422]
  We propose attentional prototype inference (API), a probabilistic latent variable framework for few-shot segmentation. We define a global latent variable to represent the prototype of each object category, which we model as a probabilistic distribution. We conduct extensive experiments on four benchmarks, where our proposal obtains at least competitive and often better performance than state-of-the-art prototype-based methods.
  arXiv  Detail & Related papers  (2021-05-14T06:58:44Z)
• Prototype Completion with Primitive Knowledge for Few-Shot Learning [20.449056536438658]
  Few-shot learning is a challenging task, which aims to learn a classifier for novel classes with few examples. Pre-training based meta-learning methods effectively tackle the problem by pre-training a feature extractor and then fine-tuning it through the nearest centroid based meta-learning. We propose a novel prototype completion based meta-learning framework.
  arXiv  Detail & Related papers  (2020-09-10T16:09:34Z)
• Learning Sparse Prototypes for Text Generation [120.38555855991562]
  Prototype-driven text generation is inefficient at test time as a result of needing to store and index the entire training corpus. We propose a novel generative model that automatically learns a sparse prototype support set that achieves strong language modeling performance. In experiments, our model outperforms previous prototype-driven language models while achieving up to a 1000x memory reduction.
  arXiv  Detail & Related papers  (2020-06-29T19:41:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.