Out of Distribution Detection via Neural Network Anchoring
- URL: http://arxiv.org/abs/2207.04125v1
- Date: Fri, 8 Jul 2022 21:01:09 GMT
- Title: Out of Distribution Detection via Neural Network Anchoring
- Authors: Rushil Anirudh, Jayaraman J. Thiagarajan
- Abstract summary: We exploit heteroscedastic temperature scaling as a calibration strategy for out of distribution (OOD) detection.
We propose a new training strategy called anchoring that can estimate appropriate temperature values for each sample.
In contrast to some of the best-performing OOD detection approaches, our method does not require exposure to additional outlier datasets.
- Score: 38.36467447555689
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Our goal in this paper is to exploit heteroscedastic temperature scaling as a
calibration strategy for out of distribution (OOD) detection.
Heteroscedasticity here refers to the fact that the optimal temperature
parameter for each sample can be different, as opposed to conventional
approaches that use the same value for the entire distribution. To enable this,
we propose a new training strategy called anchoring that can estimate
appropriate temperature values for each sample, leading to state-of-the-art OOD
detection performance across several benchmarks. Using NTK theory, we show that
this temperature function estimate is closely linked to the epistemic
uncertainty of the classifier, which explains its behavior. In contrast to some
of the best-performing OOD detection approaches, our method does not require
exposure to additional outlier datasets, custom calibration objectives, or
model ensembling. Through empirical studies with different OOD detection
settings -- far OOD, near OOD, and semantically coherent OOD - we establish a
highly effective OOD detection approach. Code and models can be accessed here
-- https://github.com/rushilanirudh/AMP
Related papers
- OAL: Enhancing OOD Detection Using Latent Diffusion [5.357756138014614]
Outlier Aware Learning (OAL) framework synthesizes OOD training data directly in the latent space.
We introduce a mutual information-based contrastive learning approach that amplifies the distinction between In-Distribution (ID) and collected OOD features.
arXiv Detail & Related papers (2024-06-24T11:01:43Z) - Model-free Test Time Adaptation for Out-Of-Distribution Detection [62.49795078366206]
We propose a Non-Parametric Test Time textbfAdaptation framework for textbfDistribution textbfDetection (abbr)
abbr utilizes online test samples for model adaptation during testing, enhancing adaptability to changing data distributions.
We demonstrate the effectiveness of abbr through comprehensive experiments on multiple OOD detection benchmarks.
arXiv Detail & Related papers (2023-11-28T02:00:47Z) - General-Purpose Multi-Modal OOD Detection Framework [5.287829685181842]
Out-of-distribution (OOD) detection identifies test samples that differ from the training data, which is critical to ensuring the safety and reliability of machine learning (ML) systems.
We propose a general-purpose weakly-supervised OOD detection framework, called WOOD, that combines a binary classifier and a contrastive learning component.
We evaluate the proposed WOOD model on multiple real-world datasets, and the experimental results demonstrate that the WOOD model outperforms the state-of-the-art methods for multi-modal OOD detection.
arXiv Detail & Related papers (2023-07-24T18:50:49Z) - Out-of-distribution Detection with Implicit Outlier Transformation [72.73711947366377]
Outlier exposure (OE) is powerful in out-of-distribution (OOD) detection.
We propose a novel OE-based approach that makes the model perform well for unseen OOD situations.
arXiv Detail & Related papers (2023-03-09T04:36:38Z) - Unsupervised Evaluation of Out-of-distribution Detection: A Data-centric
Perspective [55.45202687256175]
Out-of-distribution (OOD) detection methods assume that they have test ground truths, i.e., whether individual test samples are in-distribution (IND) or OOD.
In this paper, we are the first to introduce the unsupervised evaluation problem in OOD detection.
We propose three methods to compute Gscore as an unsupervised indicator of OOD detection performance.
arXiv Detail & Related papers (2023-02-16T13:34:35Z) - On the Usefulness of Deep Ensemble Diversity for Out-of-Distribution
Detection [7.221206118679026]
The ability to detect Out-of-Distribution (OOD) data is important in safety-critical applications of deep learning.
An existing intuition in the literature is that the diversity of Deep Ensemble predictions indicates distributional shift.
We show experimentally that this intuition is not valid on ImageNet-scale OOD detection.
arXiv Detail & Related papers (2022-07-15T15:02:38Z) - WOOD: Wasserstein-based Out-of-Distribution Detection [6.163329453024915]
Training data for deep-neural-network-based classifiers are usually assumed to be sampled from the same distribution.
When part of the test samples are drawn from a distribution that is far away from that of the training samples, the trained neural network has a tendency to make high confidence predictions for these OOD samples.
We propose a Wasserstein-based out-of-distribution detection (WOOD) method to overcome these challenges.
arXiv Detail & Related papers (2021-12-13T02:35:15Z) - Exploring Covariate and Concept Shift for Detection and Calibration of
Out-of-Distribution Data [77.27338842609153]
characterization reveals that sensitivity to each type of shift is important to the detection and confidence calibration of OOD data.
We propose a geometrically-inspired method to improve OOD detection under both shifts with only in-distribution data.
We are the first to propose a method that works well across both OOD detection and calibration and under different types of shifts.
arXiv Detail & Related papers (2021-10-28T15:42:55Z) - Robust Out-of-distribution Detection for Neural Networks [51.19164318924997]
We show that existing detection mechanisms can be extremely brittle when evaluating on in-distribution and OOD inputs.
We propose an effective algorithm called ALOE, which performs robust training by exposing the model to both adversarially crafted inlier and outlier examples.
arXiv Detail & Related papers (2020-03-21T17:46:28Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.