A statistical theory of out-of-distribution detection

• URL: http://arxiv.org/abs/2102.12959v1
• Date: Wed, 24 Feb 2021 12:35:43 GMT
• Title: A statistical theory of out-of-distribution detection
• Authors: Xi Wang, Laurence Aitchison
• Abstract summary: We introduce a principled approach to detecting out-of-distribution data by exploiting a connection to data curation. In data curation, we exclude ambiguous or difficult-to-classify input points from the dataset, and these excluded points are by definition OOD. We can therefore obtain the likelihood for OOD points by using a principled generative model of data-curation.
• Score: 26.928175726673615
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce a principled approach to detecting out-of-distribution (OOD) data by exploiting a connection to data curation. In data curation, we exclude ambiguous or difficult-to-classify input points from the dataset, and these excluded points are by definition OOD. We can therefore obtain the likelihood for OOD points by using a principled generative model of data-curation initially developed to explain the cold-posterior effect in Bayesian neural networks (Aitchison 2020). This model gives higher OOD probabilities when predictive uncertainty is higher and can be trained using maximum-likelihood jointly over the in-distribution and OOD points. This approach gives superior performance to past methods that did not provide a probability for OOD points, and therefore could not be trained using maximum-likelihood.

Related papers

• Out-of-distribution Detection Learning with Unreliable Out-of-distribution Sources [73.28967478098107]
  Out-of-distribution (OOD) detection discerns OOD data where the predictor cannot make valid predictions as in-distribution (ID) data. It is typically hard to collect real out-of-distribution (OOD) data for training a predictor capable of discerning OOD patterns. We propose a data generation-based learning method named Auxiliary Task-based OOD Learning (ATOL) that can relieve the mistaken OOD generation.
  arXiv  Detail & Related papers  (2023-11-06T16:26:52Z)
• LINe: Out-of-Distribution Detection by Leveraging Important Neurons [15.797257361788812]
  We introduce a new aspect for analyzing the difference in model outputs between in-distribution data and OOD data. We propose a novel method, Leveraging Important Neurons (LINe), for post-hoc Out of distribution detection.
  arXiv  Detail & Related papers  (2023-03-24T13:49:05Z)
• Energy-based Out-of-Distribution Detection for Graph Neural Networks [76.0242218180483]
  We propose a simple, powerful and efficient OOD detection model for GNN-based learning on graphs, which we call GNNSafe. GNNSafe achieves up to $17.0%$ AUROC improvement over state-of-the-arts and it could serve as simple yet strong baselines in such an under-developed area.
  arXiv  Detail & Related papers  (2023-02-06T16:38:43Z)
• Entropic Issues in Likelihood-Based OOD Detection [14.612834877367046]
  We provide a novel perspective on the phenomenon, decomposing the average likelihood into a KL divergence term and an entropy term. We argue that the latter can explain the curious OOD behaviour, suppressing likelihood values on datasets with higher entropy.
  arXiv  Detail & Related papers  (2021-09-22T15:32:57Z)
• Understanding Failures in Out-of-Distribution Detection with Deep Generative Models [22.11487118547924]
  We prove that no method can guarantee performance beyond random chance without assumptions on which out-distributions are relevant. We highlight the consequences implied by assuming support overlap between in- and out-distributions. Our results suggest that estimation error is a more plausible explanation than the misalignment between likelihood-based OOD detection and out-distributions of interest.
  arXiv  Detail & Related papers  (2021-07-14T18:00:11Z)
• Provably Robust Detection of Out-of-distribution Data (almost) for free [124.14121487542613]
  Deep neural networks are known to produce highly overconfident predictions on out-of-distribution (OOD) data. In this paper we propose a novel method where from first principles we combine a certifiable OOD detector with a standard classifier into an OOD aware classifier. In this way we achieve the best of two worlds: certifiably adversarially robust OOD detection, even for OOD samples close to the in-distribution, without loss in prediction accuracy and close to state-of-the-art OOD detection performance for non-manipulated OOD data.
  arXiv  Detail & Related papers  (2021-06-08T11:40:49Z)
• Learn what you can't learn: Regularized Ensembles for Transductive Out-of-distribution Detection [76.39067237772286]
  We show that current out-of-distribution (OOD) detection algorithms for neural networks produce unsatisfactory results in a variety of OOD detection scenarios. This paper studies how such "hard" OOD scenarios can benefit from adjusting the detection method after observing a batch of the test data. We propose a novel method that uses an artificial labeling scheme for the test data and regularization to obtain ensembles of models that produce contradictory predictions only on the OOD samples in a test batch.
  arXiv  Detail & Related papers  (2020-12-10T16:55:13Z)
• Out-of-Distribution Detection for Automotive Perception [58.34808836642603]
  Neural networks (NNs) are widely used for object classification in autonomous driving. NNs can fail on input data not well represented by the training dataset, known as out-of-distribution (OOD) data. This paper presents a method for determining whether inputs are OOD, which does not require OOD data during training and does not increase the computational cost of inference.
  arXiv  Detail & Related papers  (2020-11-03T01:46:35Z)
• Certifiably Adversarially Robust Detection of Out-of-Distribution Data [111.67388500330273]
  We aim for certifiable worst case guarantees for OOD detection by enforcing low confidence at the OOD point. We show that non-trivial bounds on the confidence for OOD data generalizing beyond the OOD dataset seen at training time are possible.
  arXiv  Detail & Related papers  (2020-07-16T17:16:47Z)
• Posterior Network: Uncertainty Estimation without OOD Samples via Density-Based Pseudo-Counts [33.45069308137142]
  Posterior Network (PostNet) predicts an individual closed-form posterior distribution over predicted probabilites for any input sample. PostNet achieves state-of-the art results in OOD detection and in uncertainty calibration under dataset shifts.
  arXiv  Detail & Related papers  (2020-06-16T15:16:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.