Self-Supervised Anomaly Detection by Self-Distillation and Negative
Sampling
- URL: http://arxiv.org/abs/2201.06378v1
- Date: Mon, 17 Jan 2022 12:33:14 GMT
- Title: Self-Supervised Anomaly Detection by Self-Distillation and Negative
Sampling
- Authors: Nima Rafiee, Rahil Gholamipoorfard, Nikolas Adaloglou, Simon Jaxy,
Julius Ramakers, Markus Kollmann
- Abstract summary: We show that self-distillation of the in-distribution training set together with contrasting against negative examples strongly improves OOD detection.
We observe that by leveraging negative samples, which keep the statistics of low-level features while changing the high-level semantics, higher average detection performance is obtained.
- Score: 1.304892050913381
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Detecting whether examples belong to a given in-distribution or are
Out-Of-Distribution (OOD) requires identifying features specific to the
in-distribution. In the absence of labels, these features can be learned by
self-supervised techniques under the generic assumption that the most abstract
features are those which are statistically most over-represented in comparison
to other distributions from the same domain. In this work, we show that
self-distillation of the in-distribution training set together with contrasting
against negative examples derived from shifting transformation of auxiliary
data strongly improves OOD detection. We find that this improvement depends on
how the negative samples are generated. In particular, we observe that by
leveraging negative samples, which keep the statistics of low-level features
while changing the high-level semantics, higher average detection performance
is obtained. Furthermore, good negative sampling strategies can be identified
from the sensitivity of the OOD detection score. The efficiency of our approach
is demonstrated across a diverse range of OOD detection problems, setting new
benchmarks for unsupervised OOD detection in the visual domain.
Related papers
- Rethinking the Evaluation of Out-of-Distribution Detection: A Sorites Paradox [70.57120710151105]
Most existing out-of-distribution (OOD) detection benchmarks classify samples with novel labels as the OOD data.
Some marginal OOD samples actually have close semantic contents to the in-distribution (ID) sample, which makes determining the OOD sample a Sorites Paradox.
We construct a benchmark named Incremental Shift OOD (IS-OOD) to address the issue.
arXiv Detail & Related papers (2024-06-14T09:27:56Z) - Toward a Realistic Benchmark for Out-of-Distribution Detection [3.8038269045375515]
We introduce a comprehensive benchmark for OOD detection based on ImageNet and Places365.
Several techniques can be used to determine which classes should be considered in-distribution, yielding benchmarks with varying properties.
arXiv Detail & Related papers (2024-04-16T11:29:43Z) - LINe: Out-of-Distribution Detection by Leveraging Important Neurons [15.797257361788812]
We introduce a new aspect for analyzing the difference in model outputs between in-distribution data and OOD data.
We propose a novel method, Leveraging Important Neurons (LINe), for post-hoc Out of distribution detection.
arXiv Detail & Related papers (2023-03-24T13:49:05Z) - Robustness to Spurious Correlations Improves Semantic
Out-of-Distribution Detection [24.821151013905865]
Methods which utilize the outputs or feature representations of predictive models have emerged as promising approaches for out-of-distribution (OOD) detection of image inputs.
We provide a possible explanation for SN-OOD detection failures and propose nuisance-aware OOD detection to address them.
arXiv Detail & Related papers (2023-02-08T15:28:33Z) - Energy-based Out-of-Distribution Detection for Graph Neural Networks [76.0242218180483]
We propose a simple, powerful and efficient OOD detection model for GNN-based learning on graphs, which we call GNNSafe.
GNNSafe achieves up to $17.0%$ AUROC improvement over state-of-the-arts and it could serve as simple yet strong baselines in such an under-developed area.
arXiv Detail & Related papers (2023-02-06T16:38:43Z) - Breaking Down Out-of-Distribution Detection: Many Methods Based on OOD
Training Data Estimate a Combination of the Same Core Quantities [104.02531442035483]
The goal of this paper is to recognize common objectives as well as to identify the implicit scoring functions of different OOD detection methods.
We show that binary discrimination between in- and (different) out-distributions is equivalent to several distinct formulations of the OOD detection problem.
We also show that the confidence loss which is used by Outlier Exposure has an implicit scoring function which differs in a non-trivial fashion from the theoretically optimal scoring function.
arXiv Detail & Related papers (2022-06-20T16:32:49Z) - WOOD: Wasserstein-based Out-of-Distribution Detection [6.163329453024915]
Training data for deep-neural-network-based classifiers are usually assumed to be sampled from the same distribution.
When part of the test samples are drawn from a distribution that is far away from that of the training samples, the trained neural network has a tendency to make high confidence predictions for these OOD samples.
We propose a Wasserstein-based out-of-distribution detection (WOOD) method to overcome these challenges.
arXiv Detail & Related papers (2021-12-13T02:35:15Z) - Label Smoothed Embedding Hypothesis for Out-of-Distribution Detection [72.35532598131176]
We propose an unsupervised method to detect OOD samples using a $k$-NN density estimate.
We leverage a recent insight about label smoothing, which we call the emphLabel Smoothed Embedding Hypothesis
We show that our proposal outperforms many OOD baselines and also provide new finite-sample high-probability statistical results.
arXiv Detail & Related papers (2021-02-09T21:04:44Z) - Learn what you can't learn: Regularized Ensembles for Transductive
Out-of-distribution Detection [76.39067237772286]
We show that current out-of-distribution (OOD) detection algorithms for neural networks produce unsatisfactory results in a variety of OOD detection scenarios.
This paper studies how such "hard" OOD scenarios can benefit from adjusting the detection method after observing a batch of the test data.
We propose a novel method that uses an artificial labeling scheme for the test data and regularization to obtain ensembles of models that produce contradictory predictions only on the OOD samples in a test batch.
arXiv Detail & Related papers (2020-12-10T16:55:13Z) - Robust Out-of-distribution Detection for Neural Networks [51.19164318924997]
We show that existing detection mechanisms can be extremely brittle when evaluating on in-distribution and OOD inputs.
We propose an effective algorithm called ALOE, which performs robust training by exposing the model to both adversarially crafted inlier and outlier examples.
arXiv Detail & Related papers (2020-03-21T17:46:28Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.