Out-of-Distribution Detection & Applications With Ablated Learned
Temperature Energy
- URL: http://arxiv.org/abs/2401.12129v1
- Date: Mon, 22 Jan 2024 17:11:01 GMT
- Title: Out-of-Distribution Detection & Applications With Ablated Learned
Temperature Energy
- Authors: Will LeVine, Benjamin Pikus, Jacob Phillips, Berk Norman, Fernando
Amat Gil, Sean Hendryx
- Abstract summary: We introduce Ablated Learned Temperature Energy (or "AbeT" for short)
AbeT lowers the False Positive Rate at $95%$ True Positive Rate (FPR@95) by $35.39%$ in classification.
We additionally provide empirical insights as to how our model learns to distinguish between In-Distribution (ID) and Out-of-Distribution (OOD) samples.
- Score: 40.02298833349518
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: As deep neural networks become adopted in high-stakes domains, it is crucial
to be able to identify when inference inputs are Out-of-Distribution (OOD) so
that users can be alerted of likely drops in performance and calibration
despite high confidence. Among many others, existing methods use the following
two scores to do so without training on any apriori OOD examples: a learned
temperature and an energy score. In this paper we introduce Ablated Learned
Temperature Energy (or "AbeT" for short), a method which combines these prior
methods in novel ways with effective modifications. Due to these contributions,
AbeT lowers the False Positive Rate at $95\%$ True Positive Rate (FPR@95) by
$35.39\%$ in classification (averaged across all ID and OOD datasets measured)
compared to state of the art without training networks in multiple stages or
requiring hyperparameters or test-time backward passes. We additionally provide
empirical insights as to how our model learns to distinguish between
In-Distribution (ID) and OOD samples while only being explicitly trained on ID
samples via exposure to misclassified ID examples at training time. Lastly, we
show the efficacy of our method in identifying predicted bounding boxes and
pixels corresponding to OOD objects in object detection and semantic
segmentation, respectively - with an AUROC increase of $5.15\%$ in object
detection and both a decrease in FPR@95 of $41.48\%$ and an increase in AUPRC
of $34.20\%$ on average in semantic segmentation compared to previous state of
the art.
Related papers
- Gradient-Regularized Out-of-Distribution Detection [28.542499196417214]
One of the challenges for neural networks in real-life applications is the overconfident errors these models make when the data is not from the original training distribution.
We propose the idea of leveraging the information embedded in the gradient of the loss function during training to enable the network to learn a desired OOD score for each sample.
We also develop a novel energy-based sampling method to allow the network to be exposed to more informative OOD samples during the training phase.
arXiv Detail & Related papers (2024-04-18T17:50:23Z) - Model-free Test Time Adaptation for Out-Of-Distribution Detection [62.49795078366206]
We propose a Non-Parametric Test Time textbfAdaptation framework for textbfDistribution textbfDetection (abbr)
abbr utilizes online test samples for model adaptation during testing, enhancing adaptability to changing data distributions.
We demonstrate the effectiveness of abbr through comprehensive experiments on multiple OOD detection benchmarks.
arXiv Detail & Related papers (2023-11-28T02:00:47Z) - Scaling for Training Time and Post-hoc Out-of-distribution Detection
Enhancement [41.650761556671775]
In this paper, we offer insights and analyses of recent state-of-the-art out-of-distribution (OOD) detection methods.
We demonstrate that activation pruning has a detrimental effect on OOD detection, while activation scaling enhances it.
We achieve AUROC scores of +1.85% for near-OOD and +0.74% for far-OOD datasets on the OpenOOD v1.5 ImageNet-1K benchmark.
arXiv Detail & Related papers (2023-09-30T02:10:54Z) - HAct: Out-of-Distribution Detection with Neural Net Activation
Histograms [7.795929277007233]
We propose a novel descriptor, HAct, for OOD detection, that is, probability distributions (approximated by histograms) of output values of neural network layers under the influence of incoming data.
We demonstrate that HAct is significantly more accurate than state-of-the-art in OOD detection on multiple image classification benchmarks.
arXiv Detail & Related papers (2023-09-09T16:22:18Z) - Diffusion Denoised Smoothing for Certified and Adversarial Robust
Out-Of-Distribution Detection [6.247268652296234]
We present a novel approach for certifying the robustness of OOD detection within a $ell$-norm around the input.
We improve current techniques for detecting adversarial attacks on OOD samples, while providing high levels of certified and adversarial robustness on in-distribution samples.
arXiv Detail & Related papers (2023-03-27T07:52:58Z) - Partial and Asymmetric Contrastive Learning for Out-of-Distribution
Detection in Long-Tailed Recognition [80.07843757970923]
We show that existing OOD detection methods suffer from significant performance degradation when the training set is long-tail distributed.
We propose Partial and Asymmetric Supervised Contrastive Learning (PASCL), which explicitly encourages the model to distinguish between tail-class in-distribution samples and OOD samples.
Our method outperforms previous state-of-the-art method by $1.29%$, $1.45%$, $0.69%$ anomaly detection false positive rate (FPR) and $3.24%$, $4.06%$, $7.89%$ in-distribution
arXiv Detail & Related papers (2022-07-04T01:53:07Z) - Breaking Down Out-of-Distribution Detection: Many Methods Based on OOD
Training Data Estimate a Combination of the Same Core Quantities [104.02531442035483]
The goal of this paper is to recognize common objectives as well as to identify the implicit scoring functions of different OOD detection methods.
We show that binary discrimination between in- and (different) out-distributions is equivalent to several distinct formulations of the OOD detection problem.
We also show that the confidence loss which is used by Outlier Exposure has an implicit scoring function which differs in a non-trivial fashion from the theoretically optimal scoring function.
arXiv Detail & Related papers (2022-06-20T16:32:49Z) - Label Smoothed Embedding Hypothesis for Out-of-Distribution Detection [72.35532598131176]
We propose an unsupervised method to detect OOD samples using a $k$-NN density estimate.
We leverage a recent insight about label smoothing, which we call the emphLabel Smoothed Embedding Hypothesis
We show that our proposal outperforms many OOD baselines and also provide new finite-sample high-probability statistical results.
arXiv Detail & Related papers (2021-02-09T21:04:44Z) - Learn what you can't learn: Regularized Ensembles for Transductive
Out-of-distribution Detection [76.39067237772286]
We show that current out-of-distribution (OOD) detection algorithms for neural networks produce unsatisfactory results in a variety of OOD detection scenarios.
This paper studies how such "hard" OOD scenarios can benefit from adjusting the detection method after observing a batch of the test data.
We propose a novel method that uses an artificial labeling scheme for the test data and regularization to obtain ensembles of models that produce contradictory predictions only on the OOD samples in a test batch.
arXiv Detail & Related papers (2020-12-10T16:55:13Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.