Prototype-based interpretation of the functionality of neurons in
winner-take-all neural networks
- URL: http://arxiv.org/abs/2008.08750v1
- Date: Thu, 20 Aug 2020 03:15:37 GMT
- Title: Prototype-based interpretation of the functionality of neurons in
winner-take-all neural networks
- Authors: Ramin Zarei Sabzevar, Kamaledin Ghiasi-Shirazi, Ahad Harati
- Abstract summary: Prototype-based learning (PbL) using a winner-take-all (WTA) network based on minimum Euclidean distance (ED-WTA) is an intuitive approach to multiclass classification.
We propose a novel training algorithm for the $pm$ED-WTA network, which cleverly switches between updating the positive and negative prototypes.
We show that the proposed $pm$ED-WTA method constructs highly interpretable prototypes that can be successfully used for detecting and adversarial examples.
- Score: 1.418033127602866
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Prototype-based learning (PbL) using a winner-take-all (WTA) network based on
minimum Euclidean distance (ED-WTA) is an intuitive approach to multiclass
classification. By constructing meaningful class centers, PbL provides higher
interpretability and generalization than hyperplane-based learning (HbL)
methods based on maximum Inner Product (IP-WTA) and can efficiently detect and
reject samples that do not belong to any classes. In this paper, we first prove
the equivalence of IP-WTA and ED-WTA from a representational point of view.
Then, we show that naively using this equivalence leads to unintuitive ED-WTA
networks in which the centers have high distances to data that they represent.
We propose $\pm$ED-WTA which models each neuron with two prototypes: one
positive prototype representing samples that are modeled by this neuron and a
negative prototype representing the samples that are erroneously won by that
neuron during training. We propose a novel training algorithm for the
$\pm$ED-WTA network, which cleverly switches between updating the positive and
negative prototypes and is essential to the emergence of interpretable
prototypes. Unexpectedly, we observed that the negative prototype of each
neuron is indistinguishably similar to the positive one. The rationale behind
this observation is that the training data that are mistaken with a prototype
are indeed similar to it. The main finding of this paper is this interpretation
of the functionality of neurons as computing the difference between the
distances to a positive and a negative prototype, which is in agreement with
the BCM theory. In our experiments, we show that the proposed $\pm$ED-WTA
method constructs highly interpretable prototypes that can be successfully used
for detecting outlier and adversarial examples.
Related papers
- This Probably Looks Exactly Like That: An Invertible Prototypical Network [8.957872207471311]
Prototypical neural networks represent an exciting way forward in realizing human-comprehensible machine learning without concept annotations.
We find that reliance on indirect interpretation functions for prototypical explanations imposes a severe limit on prototypes' informative power.
We propose one such model, called ProtoFlow, by composing a normalizing flow with Gaussian mixture models.
arXiv Detail & Related papers (2024-07-16T21:51:02Z) - Negative Prototypes Guided Contrastive Learning for WSOD [8.102080369924911]
Weakly Supervised Object Detection (WSOD) with only image-level annotation has recently attracted wide attention.
We propose the Negative Prototypes Guided Contrastive learning architecture.
Our proposed method achieves the state-of-the-art performance.
arXiv Detail & Related papers (2024-06-04T08:16:26Z) - Learning with Mixture of Prototypes for Out-of-Distribution Detection [25.67011646236146]
Out-of-distribution (OOD) detection aims to detect testing samples far away from the in-distribution (ID) training data.
We propose PrototypicAl Learning with a Mixture of prototypes (PALM) which models each class with multiple prototypes to capture the sample diversities.
Our method achieves state-of-the-art average AUROC performance of 93.82 on the challenging CIFAR-100 benchmark.
arXiv Detail & Related papers (2024-02-05T00:52:50Z) - Rethinking Person Re-identification from a Projection-on-Prototypes
Perspective [84.24742313520811]
Person Re-IDentification (Re-ID) as a retrieval task, has achieved tremendous development over the past decade.
We propose a new baseline ProNet, which innovatively reserves the function of the classifier at the inference stage.
Experiments on four benchmarks demonstrate that our proposed ProNet is simple yet effective, and significantly beats previous baselines.
arXiv Detail & Related papers (2023-08-21T13:38:10Z) - Stochastic Deep Networks with Linear Competing Units for Model-Agnostic
Meta-Learning [4.97235247328373]
This work addresses meta-learning (ML) by considering deep networks with local winner-takes-all (LWTA) activations.
This type of network units results in sparse representations from each model layer, as the units are organized into blocks where only one unit generates a non-zero output.
Our approach produces state-of-the-art predictive accuracy on few-shot image classification and regression experiments, as well as reduced predictive error on an active learning setting.
arXiv Detail & Related papers (2022-08-02T16:19:54Z) - Large-Margin Representation Learning for Texture Classification [67.94823375350433]
This paper presents a novel approach combining convolutional layers (CLs) and large-margin metric learning for training supervised models on small datasets for texture classification.
The experimental results on texture and histopathologic image datasets have shown that the proposed approach achieves competitive accuracy with lower computational cost and faster convergence when compared to equivalent CNNs.
arXiv Detail & Related papers (2022-06-17T04:07:45Z) - TTAPS: Test-Time Adaption by Aligning Prototypes using Self-Supervision [70.05605071885914]
We propose a novel modification of the self-supervised training algorithm SwAV that adds the ability to adapt to single test samples.
We show the success of our method on the common benchmark dataset CIFAR10-C.
arXiv Detail & Related papers (2022-05-18T05:43:06Z) - Rethinking Semantic Segmentation: A Prototype View [126.59244185849838]
We present a nonparametric semantic segmentation model based on non-learnable prototypes.
Our framework yields compelling results over several datasets.
We expect this work will provoke a rethink of the current de facto semantic segmentation model design.
arXiv Detail & Related papers (2022-03-28T21:15:32Z) - Dual Prototypical Contrastive Learning for Few-shot Semantic
Segmentation [55.339405417090084]
We propose a dual prototypical contrastive learning approach tailored to the few-shot semantic segmentation (FSS) task.
The main idea is to encourage the prototypes more discriminative by increasing inter-class distance while reducing intra-class distance in prototype feature space.
We demonstrate that the proposed dual contrastive learning approach outperforms state-of-the-art FSS methods on PASCAL-5i and COCO-20i datasets.
arXiv Detail & Related papers (2021-11-09T08:14:50Z) - ProtoryNet - Interpretable Text Classification Via Prototype
Trajectories [4.768286204382179]
We propose a novel interpretable deep neural network for text classification, called ProtoryNet.
ProtoryNet makes a prediction by finding the most similar prototype for each sentence in a text sequence.
After prototype pruning, the resulting ProtoryNet models only need less than or around 20 prototypes for all datasets.
arXiv Detail & Related papers (2020-07-03T16:00:26Z) - Prototypical Contrastive Learning of Unsupervised Representations [171.3046900127166]
Prototypical Contrastive Learning (PCL) is an unsupervised representation learning method.
PCL implicitly encodes semantic structures of the data into the learned embedding space.
PCL outperforms state-of-the-art instance-wise contrastive learning methods on multiple benchmarks.
arXiv Detail & Related papers (2020-05-11T09:53:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.