No Shifted Augmentations (NSA): compact distributions for robust
self-supervised Anomaly Detection
- URL: http://arxiv.org/abs/2203.10344v1
- Date: Sat, 19 Mar 2022 15:55:32 GMT
- Title: No Shifted Augmentations (NSA): compact distributions for robust
self-supervised Anomaly Detection
- Authors: Mohamed Yousef, Marcel Ackermann, Unmesh Kurup, Tom Bishop
- Abstract summary: Unsupervised Anomaly detection (AD) requires building a notion of normalcy, distinguishing in-distribution (ID) and out-of-distribution (OOD) data.
We investigate how the emph geometrical compactness of the ID feature distribution makes isolating and detecting outliers easier.
We propose novel architectural modifications to the self-supervised feature learning step, that enable such compact distributions for ID data to be learned.
- Score: 4.243926243206826
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Unsupervised Anomaly detection (AD) requires building a notion of normalcy,
distinguishing in-distribution (ID) and out-of-distribution (OOD) data, using
only available ID samples. Recently, large gains were made on this task for the
domain of natural images using self-supervised contrastive feature learning as
a first step followed by kNN or traditional one-class classifiers for feature
scoring. Learned representations that are non-uniformly distributed on the unit
hypersphere have been shown to be beneficial for this task. We go a step
further and investigate how the \emph {geometrical compactness} of the ID
feature distribution makes isolating and detecting outliers easier, especially
in the realistic situation when ID training data is polluted (i.e. ID data
contains some OOD data that is used for learning the feature extractor
parameters). We propose novel architectural modifications to the
self-supervised feature learning step, that enable such compact distributions
for ID data to be learned. We show that the proposed modifications can be
effectively applied to most existing self-supervised objectives, with large
gains in performance. Furthermore, this improved OOD performance is obtained
without resorting to tricks such as using strongly augmented ID images (e.g. by
90 degree rotations) as proxies for the unseen OOD data, as these impose overly
prescriptive assumptions about ID data and its invariances. We perform
extensive studies on benchmark datasets for one-class OOD detection and show
state-of-the-art performance in the presence of pollution in the ID data, and
comparable performance otherwise. We also propose and extensively evaluate a
novel feature scoring technique based on the angular Mahalanobis distance, and
propose a simple and novel technique for feature ensembling during evaluation
that enables a big boost in performance at nearly zero run-time cost.
Related papers
- Exploiting Diffusion Prior for Out-of-Distribution Detection [11.11093497717038]
Out-of-distribution (OOD) detection is crucial for deploying robust machine learning models.
We present a novel approach for OOD detection that leverages the generative ability of diffusion models and the powerful feature extraction capabilities of CLIP.
arXiv Detail & Related papers (2024-06-16T23:55:25Z) - Pursuing Feature Separation based on Neural Collapse for Out-of-Distribution Detection [21.357620914949624]
In the open world, detecting out-of-distribution (OOD) data, whose labels are disjoint with those of in-distribution (ID) samples, is important for reliable deep neural networks (DNNs)
We propose a simple but effective loss called OrthLoss, which binds the features of OOD data in a subspace to the principal subspace of ID features formed by NC.
Our detection achieves SOTA performance on CIFAR benchmarks without any additional data augmentation or sampling.
arXiv Detail & Related papers (2024-05-28T04:24:38Z) - EAT: Towards Long-Tailed Out-of-Distribution Detection [55.380390767978554]
This paper addresses the challenging task of long-tailed OOD detection.
The main difficulty lies in distinguishing OOD data from samples belonging to the tail classes.
We propose two simple ideas: (1) Expanding the in-distribution class space by introducing multiple abstention classes, and (2) Augmenting the context-limited tail classes by overlaying images onto the context-rich OOD data.
arXiv Detail & Related papers (2023-12-14T13:47:13Z) - Diversified Outlier Exposure for Out-of-Distribution Detection via
Informative Extrapolation [110.34982764201689]
Out-of-distribution (OOD) detection is important for deploying reliable machine learning models on real-world applications.
Recent advances in outlier exposure have shown promising results on OOD detection via fine-tuning model with informatively sampled auxiliary outliers.
We propose a novel framework, namely, Diversified Outlier Exposure (DivOE), for effective OOD detection via informative extrapolation based on the given auxiliary outliers.
arXiv Detail & Related papers (2023-10-21T07:16:09Z) - From Global to Local: Multi-scale Out-of-distribution Detection [129.37607313927458]
Out-of-distribution (OOD) detection aims to detect "unknown" data whose labels have not been seen during the in-distribution (ID) training process.
Recent progress in representation learning gives rise to distance-based OOD detection.
We propose Multi-scale OOD DEtection (MODE), a first framework leveraging both global visual information and local region details.
arXiv Detail & Related papers (2023-08-20T11:56:25Z) - Unleashing Mask: Explore the Intrinsic Out-of-Distribution Detection
Capability [70.72426887518517]
Out-of-distribution (OOD) detection is an indispensable aspect of secure AI when deploying machine learning models in real-world applications.
We propose a novel method, Unleashing Mask, which aims to restore the OOD discriminative capabilities of the well-trained model with ID data.
Our method utilizes a mask to figure out the memorized atypical samples, and then finetune the model or prune it with the introduced mask to forget them.
arXiv Detail & Related papers (2023-06-06T14:23:34Z) - Cluster-level pseudo-labelling for source-free cross-domain facial
expression recognition [94.56304526014875]
We propose the first Source-Free Unsupervised Domain Adaptation (SFUDA) method for Facial Expression Recognition (FER)
Our method exploits self-supervised pretraining to learn good feature representations from the target data.
We validate the effectiveness of our method in four adaptation setups, proving that it consistently outperforms existing SFUDA methods when applied to FER.
arXiv Detail & Related papers (2022-10-11T08:24:50Z) - Augmenting Softmax Information for Selective Classification with
Out-of-Distribution Data [7.221206118679026]
We show that existing post-hoc methods perform quite differently compared to when evaluated only on OOD detection.
We propose a novel method for SCOD, Softmax Information Retaining Combination (SIRC), that augments softmax-based confidence scores with feature-agnostic information.
Experiments on a wide variety of ImageNet-scale datasets and convolutional neural network architectures show that SIRC is able to consistently match or outperform the baseline for SCOD.
arXiv Detail & Related papers (2022-07-15T14:39:57Z) - Training OOD Detectors in their Natural Habitats [31.565635192716712]
Out-of-distribution (OOD) detection is important for machine learning models deployed in the wild.
Recent methods use auxiliary outlier data to regularize the model for improved OOD detection.
We propose a novel framework that leverages wild mixture data -- that naturally consists of both ID and OOD samples.
arXiv Detail & Related papers (2022-02-07T15:38:39Z) - Semantically Coherent Out-of-Distribution Detection [26.224146828317277]
Current out-of-distribution (OOD) detection benchmarks are commonly built by defining one dataset as in-distribution (ID) and all others as OOD.
We re-design the benchmarks and propose the semantically coherent out-of-distribution detection (SC-OOD)
Our approach achieves state-of-the-art performance on SC-OOD benchmarks.
arXiv Detail & Related papers (2021-08-26T17:53:32Z) - Omni-supervised Facial Expression Recognition via Distilled Data [120.11782405714234]
We propose omni-supervised learning to exploit reliable samples in a large amount of unlabeled data for network training.
We experimentally verify that the new dataset can significantly improve the ability of the learned FER model.
To tackle this, we propose to apply a dataset distillation strategy to compress the created dataset into several informative class-wise images.
arXiv Detail & Related papers (2020-05-18T09:36:51Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.