Energy-based Out-of-distribution Detection

• URL: http://arxiv.org/abs/2010.03759v4
• Date: Mon, 26 Apr 2021 04:59:58 GMT
• Title: Energy-based Out-of-distribution Detection
• Authors: Weitang Liu, Xiaoyun Wang, John D. Owens, Yixuan Li
• Abstract summary: We propose a unified framework for OOD detection that uses an energy score. We show that energy scores better distinguish in- and out-of-distribution samples than the traditional approach. With energy-based training, our method outperforms the state-of-the-art on common benchmarks.
• Score: 24.320646820385065
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Determining whether inputs are out-of-distribution (OOD) is an essential building block for safely deploying machine learning models in the open world. However, previous methods relying on the softmax confidence score suffer from overconfident posterior distributions for OOD data. We propose a unified framework for OOD detection that uses an energy score. We show that energy scores better distinguish in- and out-of-distribution samples than the traditional approach using the softmax scores. Unlike softmax confidence scores, energy scores are theoretically aligned with the probability density of the inputs and are less susceptible to the overconfidence issue. Within this framework, energy can be flexibly used as a scoring function for any pre-trained neural classifier as well as a trainable cost function to shape the energy surface explicitly for OOD detection. On a CIFAR-10 pre-trained WideResNet, using the energy score reduces the average FPR (at TPR 95%) by 18.03% compared to the softmax confidence score. With energy-based training, our method outperforms the state-of-the-art on common benchmarks.

Related papers

• What If the Input is Expanded in OOD Detection? [77.37433624869857]
  Out-of-distribution (OOD) detection aims to identify OOD inputs from unknown classes. Various scoring functions are proposed to distinguish it from in-distribution (ID) data. We introduce a novel perspective, i.e., employing different common corruptions on the input space.
  arXiv  Detail & Related papers  (2024-10-24T06:47:28Z)
• Taming False Positives in Out-of-Distribution Detection with Human Feedback [6.192472816262214]
  We propose a mathematically grounded OOD detection framework that leverages expert feedback to emphsafely update the threshold on the fly. We provide theoretical results showing that it is guaranteed to meet the FPR constraint at all times while minimizing the use of human feedback.
  arXiv  Detail & Related papers  (2024-04-25T18:06:47Z)
• Energy-based Out-of-Distribution Detection for Graph Neural Networks [76.0242218180483]
  We propose a simple, powerful and efficient OOD detection model for GNN-based learning on graphs, which we call GNNSafe. GNNSafe achieves up to $17.0%$ AUROC improvement over state-of-the-arts and it could serve as simple yet strong baselines in such an under-developed area.
  arXiv  Detail & Related papers  (2023-02-06T16:38:43Z)
• Disentangling Confidence Score Distribution for Out-of-Domain Intent Detection with Energy-Based Learning [40.96874034407684]
  We propose a simple but strong energy-based score function to detect OOD where the energy scores of OOD samples are higher than IND samples. Comprehensive experiments and analysis prove our method helps disentangle confidence score distributions of IND and OOD data.
  arXiv  Detail & Related papers  (2022-10-17T08:19:01Z)
• RankFeat: Rank-1 Feature Removal for Out-of-distribution Detection [65.67315418971688]
  textttRankFeat is a simple yet effective textttpost hoc approach for OOD detection. textttRankFeat achieves the emphstate-of-the-art performance and reduces the average false positive rate (FPR95) by 17.90% compared with the previous best method.
  arXiv  Detail & Related papers  (2022-09-18T16:01:31Z)
• Distribution Calibration for Out-of-Domain Detection with Bayesian Approximation [35.34001858858684]
  Out-of-Domain (OOD) detection is a key component in a task-oriented dialog system. Previous softmax-based detection algorithms are proved to be overconfident for OOD samples. We propose a Bayesian OOD detection framework to calibrate distribution uncertainty using Monte-Carlo Dropout.
  arXiv  Detail & Related papers  (2022-09-14T13:04:09Z)
• Semantic Driven Energy based Out-of-Distribution Detection [0.4640835690336652]
  Energy based OOD methods have proved to be promising and achieved impressive performance. We propose semantic driven energy based method, which is an end-to-end trainable system and easy to optimize. We find that, our novel approach enhances outlier detection and achieve state-of-the-art as an energy-based model on common benchmarks.
  arXiv  Detail & Related papers  (2022-08-23T07:40:34Z)
• Energy-based Unknown Intent Detection with Data Manipulation [48.76465739088016]
  Unknown intent detection aims to identify the out-of-distribution utterance whose intent has never appeared in the training set. We propose using energy scores for this task as the energy score is theoretically aligned with the density of the input.
  arXiv  Detail & Related papers  (2021-07-27T01:32:23Z)
• Understanding Softmax Confidence and Uncertainty [95.71801498763216]
  It is often remarked that neural networks fail to increase their uncertainty when predicting on data far from the training distribution. Yet naively using softmax confidence as a proxy for uncertainty achieves modest success in tasks exclusively testing for this. This paper investigates this contradiction, identifying two implicit biases that do encourage softmax confidence to correlate with uncertainty.
  arXiv  Detail & Related papers  (2021-06-09T10:37:29Z)
• Provably Robust Detection of Out-of-distribution Data (almost) for free [124.14121487542613]
  Deep neural networks are known to produce highly overconfident predictions on out-of-distribution (OOD) data. In this paper we propose a novel method where from first principles we combine a certifiable OOD detector with a standard classifier into an OOD aware classifier. In this way we achieve the best of two worlds: certifiably adversarially robust OOD detection, even for OOD samples close to the in-distribution, without loss in prediction accuracy and close to state-of-the-art OOD detection performance for non-manipulated OOD data.
  arXiv  Detail & Related papers  (2021-06-08T11:40:49Z)
• Unsupervised Energy-based Out-of-distribution Detection using Stiefel-Restricted Kernel Machine [13.345005810199181]
  We propose an unsupervised energy-based OOD detector leveraging the Stiefel-Restricted Kernel Machine (St-RKM) In the experiments on standard datasets, the proposed method improves over the existing energy-based OOD detectors and deep generative models.
  arXiv  Detail & Related papers  (2021-02-16T20:46:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.