Interpreting deep learning output for out-of-distribution detection
- URL: http://arxiv.org/abs/2211.03637v1
- Date: Mon, 7 Nov 2022 15:48:08 GMT
- Title: Interpreting deep learning output for out-of-distribution detection
- Authors: Damian Matuszewski, Ida-Maria Sintorn
- Abstract summary: We develop a new method for out-of-distribution detection in deep learning networks.
The method offers an explanatory step towards understanding and interpretation of the model learning process and its output.
We demonstrate our OOD detection method on a challenging transmission electron microscopy virus image dataset.
- Score: 0.6091702876917279
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Commonly used AI networks are very self-confident in their predictions, even
when the evidence for a certain decision is dubious. The investigation of a
deep learning model output is pivotal for understanding its decision processes
and assessing its capabilities and limitations. By analyzing the distributions
of raw network output vectors, it can be observed that each class has its own
decision boundary and, thus, the same raw output value has different support
for different classes. Inspired by this fact, we have developed a new method
for out-of-distribution detection. The method offers an explanatory step beyond
simple thresholding of the softmax output towards understanding and
interpretation of the model learning process and its output. Instead of
assigning the class label of the highest logit to each new sample presented to
the network, it takes the distributions over all classes into consideration. A
probability score interpreter (PSI) is created based on the joint logit values
in relation to their respective correct vs wrong class distributions. The PSI
suggests whether the sample is likely to belong to a specific class, whether
the network is unsure, or whether the sample is likely an outlier or unknown
type for the network. The simple PSI has the benefit of being applicable on
already trained networks. The distributions for correct vs wrong class for each
output node are established by simply running the training examples through the
trained network. We demonstrate our OOD detection method on a challenging
transmission electron microscopy virus image dataset. We simulate a real-world
application in which images of virus types unknown to a trained virus
classifier, yet acquired with the same procedures and instruments, constitute
the OOD samples.
Related papers
- Verification of Neural Networks Local Differential Classification
Privacy [1.3024517678456733]
We propose a new privacy property, called local differential classification privacy (LDCP)
LDCP extends local robustness to a differential privacy setting suitable for black-box classifiers.
We propose Sphynx, an algorithm that computes an abstraction of all networks, with a high probability, from a small set of networks.
arXiv Detail & Related papers (2023-10-31T09:11:12Z) - Sampling weights of deep neural networks [1.2370077627846041]
We introduce a probability distribution, combined with an efficient sampling algorithm, for weights and biases of fully-connected neural networks.
In a supervised learning context, no iterative optimization or gradient computations of internal network parameters are needed.
We prove that sampled networks are universal approximators.
arXiv Detail & Related papers (2023-06-29T10:13:36Z) - Learning versus Refutation in Noninteractive Local Differential Privacy [133.80204506727526]
We study two basic statistical tasks in non-interactive local differential privacy (LDP): learning and refutation.
Our main result is a complete characterization of the sample complexity of PAC learning for non-interactive LDP protocols.
arXiv Detail & Related papers (2022-10-26T03:19:24Z) - Partial and Asymmetric Contrastive Learning for Out-of-Distribution
Detection in Long-Tailed Recognition [80.07843757970923]
We show that existing OOD detection methods suffer from significant performance degradation when the training set is long-tail distributed.
We propose Partial and Asymmetric Supervised Contrastive Learning (PASCL), which explicitly encourages the model to distinguish between tail-class in-distribution samples and OOD samples.
Our method outperforms previous state-of-the-art method by $1.29%$, $1.45%$, $0.69%$ anomaly detection false positive rate (FPR) and $3.24%$, $4.06%$, $7.89%$ in-distribution
arXiv Detail & Related papers (2022-07-04T01:53:07Z) - WOOD: Wasserstein-based Out-of-Distribution Detection [6.163329453024915]
Training data for deep-neural-network-based classifiers are usually assumed to be sampled from the same distribution.
When part of the test samples are drawn from a distribution that is far away from that of the training samples, the trained neural network has a tendency to make high confidence predictions for these OOD samples.
We propose a Wasserstein-based out-of-distribution detection (WOOD) method to overcome these challenges.
arXiv Detail & Related papers (2021-12-13T02:35:15Z) - Trash to Treasure: Harvesting OOD Data with Cross-Modal Matching for
Open-Set Semi-Supervised Learning [101.28281124670647]
Open-set semi-supervised learning (open-set SSL) investigates a challenging but practical scenario where out-of-distribution (OOD) samples are contained in the unlabeled data.
We propose a novel training mechanism that could effectively exploit the presence of OOD data for enhanced feature learning.
Our approach substantially lifts the performance on open-set SSL and outperforms the state-of-the-art by a large margin.
arXiv Detail & Related papers (2021-08-12T09:14:44Z) - DAAIN: Detection of Anomalous and Adversarial Input using Normalizing
Flows [52.31831255787147]
We introduce a novel technique, DAAIN, to detect out-of-distribution (OOD) inputs and adversarial attacks (AA)
Our approach monitors the inner workings of a neural network and learns a density estimator of the activation distribution.
Our model can be trained on a single GPU making it compute efficient and deployable without requiring specialized accelerators.
arXiv Detail & Related papers (2021-05-30T22:07:13Z) - Multi-Class Data Description for Out-of-distribution Detection [25.853322158250435]
Deep-MCDD is effective to detect out-of-distribution (OOD) samples as well as classify in-distribution (ID) samples.
By integrating the concept of Gaussian discriminant analysis into deep neural networks, we propose a deep learning objective to learn class-conditional distributions.
arXiv Detail & Related papers (2021-04-02T08:41:51Z) - Understanding Classifier Mistakes with Generative Models [88.20470690631372]
Deep neural networks are effective on supervised learning tasks, but have been shown to be brittle.
In this paper, we leverage generative models to identify and characterize instances where classifiers fail to generalize.
Our approach is agnostic to class labels from the training set which makes it applicable to models trained in a semi-supervised way.
arXiv Detail & Related papers (2020-10-05T22:13:21Z) - Learning with Out-of-Distribution Data for Audio Classification [60.48251022280506]
We show that detecting and relabelling certain OOD instances, rather than discarding them, can have a positive effect on learning.
The proposed method is shown to improve the performance of convolutional neural networks by a significant margin.
arXiv Detail & Related papers (2020-02-11T21:08:06Z) - Incremental Unsupervised Domain-Adversarial Training of Neural Networks [17.91571291302582]
In the context of supervised statistical learning, it is typically assumed that the training set comes from the same distribution that draws the test samples.
Here we take a different avenue and approach the problem from an incremental point of view, where the model is adapted to the new domain iteratively.
Our results report a clear improvement with respect to the non-incremental case in several datasets, also outperforming other state-of-the-art domain adaptation algorithms.
arXiv Detail & Related papers (2020-01-13T09:54:35Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.