Prototype-based interpretation of the functionality of neurons in
winner-take-all neural networks
- URL: http://arxiv.org/abs/2008.08750v1
- Date: Thu, 20 Aug 2020 03:15:37 GMT
- Title: Prototype-based interpretation of the functionality of neurons in
winner-take-all neural networks
- Authors: Ramin Zarei Sabzevar, Kamaledin Ghiasi-Shirazi, Ahad Harati
- Abstract summary: Prototype-based learning (PbL) using a winner-take-all (WTA) network based on minimum Euclidean distance (ED-WTA) is an intuitive approach to multiclass classification.
We propose a novel training algorithm for the $pm$ED-WTA network, which cleverly switches between updating the positive and negative prototypes.
We show that the proposed $pm$ED-WTA method constructs highly interpretable prototypes that can be successfully used for detecting and adversarial examples.
- Score: 1.418033127602866
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Prototype-based learning (PbL) using a winner-take-all (WTA) network based on
minimum Euclidean distance (ED-WTA) is an intuitive approach to multiclass
classification. By constructing meaningful class centers, PbL provides higher
interpretability and generalization than hyperplane-based learning (HbL)
methods based on maximum Inner Product (IP-WTA) and can efficiently detect and
reject samples that do not belong to any classes. In this paper, we first prove
the equivalence of IP-WTA and ED-WTA from a representational point of view.
Then, we show that naively using this equivalence leads to unintuitive ED-WTA
networks in which the centers have high distances to data that they represent.
We propose $\pm$ED-WTA which models each neuron with two prototypes: one
positive prototype representing samples that are modeled by this neuron and a
negative prototype representing the samples that are erroneously won by that
neuron during training. We propose a novel training algorithm for the
$\pm$ED-WTA network, which cleverly switches between updating the positive and
negative prototypes and is essential to the emergence of interpretable
prototypes. Unexpectedly, we observed that the negative prototype of each
neuron is indistinguishably similar to the positive one. The rationale behind
this observation is that the training data that are mistaken with a prototype
are indeed similar to it. The main finding of this paper is this interpretation
of the functionality of neurons as computing the difference between the
distances to a positive and a negative prototype, which is in agreement with
the BCM theory. In our experiments, we show that the proposed $\pm$ED-WTA
method constructs highly interpretable prototypes that can be successfully used
for detecting outlier and adversarial examples.
Related papers
- Sparse Prototype Network for Explainable Pedestrian Behavior Prediction [60.80524827122901]
We present Sparse Prototype Network (SPN), an explainable method designed to simultaneously predict a pedestrian's future action, trajectory, and pose.
Regularized by mono-semanticity and clustering constraints, the prototypes learn consistent and human-understandable features.
arXiv Detail & Related papers (2024-10-16T03:33:40Z) - GAProtoNet: A Multi-head Graph Attention-based Prototypical Network for Interpretable Text Classification [1.170190320889319]
We introduce GAProtoNet, a novel white-box Multi-head Graph Attention-based Prototypical Network.
Our approach achieves superior results without sacrificing the accuracy of the original black-box LMs.
Our case study and visualization of prototype clusters also demonstrate the efficiency in explaining the decisions of black-box models built with LMs.
arXiv Detail & Related papers (2024-09-20T08:15:17Z) - Enhanced Prototypical Part Network (EPPNet) For Explainable Image Classification Via Prototypes [16.528373143163275]
We introduce the Enhanced Prototypical Part Network (EPPNet) for image classification.
EPPNet achieves strong performance while discovering relevant prototypes that can be used to explain the classification results.
Our evaluations on the CUB-200-2011 dataset show that the EPPNet outperforms state-of-the-art xAI-based methods.
arXiv Detail & Related papers (2024-08-08T17:26:56Z) - Negative Prototypes Guided Contrastive Learning for WSOD [8.102080369924911]
Weakly Supervised Object Detection (WSOD) with only image-level annotation has recently attracted wide attention.
We propose the Negative Prototypes Guided Contrastive learning architecture.
Our proposed method achieves the state-of-the-art performance.
arXiv Detail & Related papers (2024-06-04T08:16:26Z) - Learning with Mixture of Prototypes for Out-of-Distribution Detection [25.67011646236146]
Out-of-distribution (OOD) detection aims to detect testing samples far away from the in-distribution (ID) training data.
We propose PrototypicAl Learning with a Mixture of prototypes (PALM) which models each class with multiple prototypes to capture the sample diversities.
Our method achieves state-of-the-art average AUROC performance of 93.82 on the challenging CIFAR-100 benchmark.
arXiv Detail & Related papers (2024-02-05T00:52:50Z) - Large-Margin Representation Learning for Texture Classification [67.94823375350433]
This paper presents a novel approach combining convolutional layers (CLs) and large-margin metric learning for training supervised models on small datasets for texture classification.
The experimental results on texture and histopathologic image datasets have shown that the proposed approach achieves competitive accuracy with lower computational cost and faster convergence when compared to equivalent CNNs.
arXiv Detail & Related papers (2022-06-17T04:07:45Z) - TTAPS: Test-Time Adaption by Aligning Prototypes using Self-Supervision [70.05605071885914]
We propose a novel modification of the self-supervised training algorithm SwAV that adds the ability to adapt to single test samples.
We show the success of our method on the common benchmark dataset CIFAR10-C.
arXiv Detail & Related papers (2022-05-18T05:43:06Z) - Rethinking Semantic Segmentation: A Prototype View [126.59244185849838]
We present a nonparametric semantic segmentation model based on non-learnable prototypes.
Our framework yields compelling results over several datasets.
We expect this work will provoke a rethink of the current de facto semantic segmentation model design.
arXiv Detail & Related papers (2022-03-28T21:15:32Z) - Dual Prototypical Contrastive Learning for Few-shot Semantic
Segmentation [55.339405417090084]
We propose a dual prototypical contrastive learning approach tailored to the few-shot semantic segmentation (FSS) task.
The main idea is to encourage the prototypes more discriminative by increasing inter-class distance while reducing intra-class distance in prototype feature space.
We demonstrate that the proposed dual contrastive learning approach outperforms state-of-the-art FSS methods on PASCAL-5i and COCO-20i datasets.
arXiv Detail & Related papers (2021-11-09T08:14:50Z) - ProtoryNet - Interpretable Text Classification Via Prototype
Trajectories [4.768286204382179]
We propose a novel interpretable deep neural network for text classification, called ProtoryNet.
ProtoryNet makes a prediction by finding the most similar prototype for each sentence in a text sequence.
After prototype pruning, the resulting ProtoryNet models only need less than or around 20 prototypes for all datasets.
arXiv Detail & Related papers (2020-07-03T16:00:26Z) - Prototypical Contrastive Learning of Unsupervised Representations [171.3046900127166]
Prototypical Contrastive Learning (PCL) is an unsupervised representation learning method.
PCL implicitly encodes semantic structures of the data into the learned embedding space.
PCL outperforms state-of-the-art instance-wise contrastive learning methods on multiple benchmarks.
arXiv Detail & Related papers (2020-05-11T09:53:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.