ProtoArgNet: Interpretable Image Classification with Super-Prototypes
and Argumentation [Technical Report]
- URL: http://arxiv.org/abs/2311.15438v1
- Date: Sun, 26 Nov 2023 21:52:47 GMT
- Title: ProtoArgNet: Interpretable Image Classification with Super-Prototypes
and Argumentation [Technical Report]
- Authors: Hamed Ayoobi, Nico Potyka, Francesca Toni
- Abstract summary: ProtoArgNet is a novel interpretable deep neural architecture for image classification.
It uses super-prototypes that combine prototypical-parts into single prototypical class representations.
It can recognise spatial relations between different prototypical-parts that are from different regions in images.
- Score: 19.35742377374265
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: We propose ProtoArgNet, a novel interpretable deep neural architecture for
image classification in the spirit of prototypical-part-learning as found, e.g.
in ProtoPNet. While earlier approaches associate every class with multiple
prototypical-parts, ProtoArgNet uses super-prototypes that combine
prototypical-parts into single prototypical class representations. Furthermore,
while earlier approaches use interpretable classification layers, e.g. logistic
regression in ProtoPNet, ProtoArgNet improves accuracy with multi-layer
perceptrons while relying upon an interpretable reading thereof based on a form
of argumentation. ProtoArgNet is customisable to user cognitive requirements by
a process of sparsification of the multi-layer perceptron/argumentation
component. Also, as opposed to other prototypical-part-learning approaches,
ProtoArgNet can recognise spatial relations between different
prototypical-parts that are from different regions in images, similar to how
CNNs capture relations between patterns recognized in earlier layers.
Related papers
- This Looks Like Those: Illuminating Prototypical Concepts Using Multiple
Visualizations [19.724372592639774]
ProtoConcepts is a method for interpretable image classification combining deep learning and case-based reasoning.
Our proposed method modifies the architecture of prototype-based networks to instead learn concepts which are visualized using multiple image patches.
Our experiments show that our this looks like those'' reasoning process can be applied as a modification to a wide range of existing prototypical image classification networks.
arXiv Detail & Related papers (2023-10-28T04:54:48Z) - Rethinking Person Re-identification from a Projection-on-Prototypes
Perspective [84.24742313520811]
Person Re-IDentification (Re-ID) as a retrieval task, has achieved tremendous development over the past decade.
We propose a new baseline ProNet, which innovatively reserves the function of the classifier at the inference stage.
Experiments on four benchmarks demonstrate that our proposed ProNet is simple yet effective, and significantly beats previous baselines.
arXiv Detail & Related papers (2023-08-21T13:38:10Z) - Prototype-based Embedding Network for Scene Graph Generation [105.97836135784794]
Current Scene Graph Generation (SGG) methods explore contextual information to predict relationships among entity pairs.
Due to the diverse visual appearance of numerous possible subject-object combinations, there is a large intra-class variation within each predicate category.
Prototype-based Embedding Network (PE-Net) models entities/predicates with prototype-aligned compact and distinctive representations.
PL is introduced to help PE-Net efficiently learn such entitypredicate matching, and Prototype Regularization (PR) is devised to relieve the ambiguous entity-predicate matching.
arXiv Detail & Related papers (2023-03-13T13:30:59Z) - Progressive Tree-Structured Prototype Network for End-to-End Image
Captioning [74.8547752611337]
We propose a novel Progressive Tree-Structured prototype Network (dubbed PTSN)
PTSN is the first attempt to narrow down the scope of prediction words with appropriate semantics by modeling the hierarchical textual semantics.
Our method achieves a new state-of-the-art performance with 144.2% (single model) and 146.5% (ensemble of 4 models) CIDEr scores on Karpathy' split and 141.4% (c5) and 143.9% (c40) CIDEr scores on the official online test server.
arXiv Detail & Related papers (2022-11-17T11:04:00Z) - Rethinking Semantic Segmentation: A Prototype View [126.59244185849838]
We present a nonparametric semantic segmentation model based on non-learnable prototypes.
Our framework yields compelling results over several datasets.
We expect this work will provoke a rethink of the current de facto semantic segmentation model design.
arXiv Detail & Related papers (2022-03-28T21:15:32Z) - Interpretable Image Classification with Differentiable Prototypes
Assignment [7.660883761395447]
We introduce ProtoPool, an interpretable image classification model with a pool of prototypes shared by the classes.
It is obtained by introducing a fully differentiable assignment of prototypes to particular classes.
We show that ProtoPool obtains state-of-the-art accuracy on the CUB-200-2011 and the Stanford Cars datasets, substantially reducing the number of prototypes.
arXiv Detail & Related papers (2021-12-06T10:03:32Z) - Deformable ProtoPNet: An Interpretable Image Classifier Using Deformable Prototypes [7.8515366468594765]
We present a deformable part network (Deformable ProtoPNet) that integrates the power of deep learning and the interpretability of case-based reasoning.
This model classifies input images by comparing them with prototypes learned during training, yielding explanations in the form of "this looks like that"
arXiv Detail & Related papers (2021-11-29T22:38:13Z) - Dual Prototypical Contrastive Learning for Few-shot Semantic
Segmentation [55.339405417090084]
We propose a dual prototypical contrastive learning approach tailored to the few-shot semantic segmentation (FSS) task.
The main idea is to encourage the prototypes more discriminative by increasing inter-class distance while reducing intra-class distance in prototype feature space.
We demonstrate that the proposed dual contrastive learning approach outperforms state-of-the-art FSS methods on PASCAL-5i and COCO-20i datasets.
arXiv Detail & Related papers (2021-11-09T08:14:50Z) - Part-aware Prototype Network for Few-shot Semantic Segmentation [50.581647306020095]
We propose a novel few-shot semantic segmentation framework based on the prototype representation.
Our key idea is to decompose the holistic class representation into a set of part-aware prototypes.
We develop a novel graph neural network model to generate and enhance the proposed part-aware prototypes.
arXiv Detail & Related papers (2020-07-13T11:03:09Z) - ProtoryNet - Interpretable Text Classification Via Prototype
Trajectories [4.768286204382179]
We propose a novel interpretable deep neural network for text classification, called ProtoryNet.
ProtoryNet makes a prediction by finding the most similar prototype for each sentence in a text sequence.
After prototype pruning, the resulting ProtoryNet models only need less than or around 20 prototypes for all datasets.
arXiv Detail & Related papers (2020-07-03T16:00:26Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.