Out of Distribution Detection via Neural Network Anchoring

• URL: http://arxiv.org/abs/2207.04125v1
• Date: Fri, 8 Jul 2022 21:01:09 GMT
• Title: Out of Distribution Detection via Neural Network Anchoring
• Authors: Rushil Anirudh, Jayaraman J. Thiagarajan
• Abstract summary: We exploit heteroscedastic temperature scaling as a calibration strategy for out of distribution (OOD) detection. We propose a new training strategy called anchoring that can estimate appropriate temperature values for each sample. In contrast to some of the best-performing OOD detection approaches, our method does not require exposure to additional outlier datasets.
• Score: 38.36467447555689
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Our goal in this paper is to exploit heteroscedastic temperature scaling as a calibration strategy for out of distribution (OOD) detection. Heteroscedasticity here refers to the fact that the optimal temperature parameter for each sample can be different, as opposed to conventional approaches that use the same value for the entire distribution. To enable this, we propose a new training strategy called anchoring that can estimate appropriate temperature values for each sample, leading to state-of-the-art OOD detection performance across several benchmarks. Using NTK theory, we show that this temperature function estimate is closely linked to the epistemic uncertainty of the classifier, which explains its behavior. In contrast to some of the best-performing OOD detection approaches, our method does not require exposure to additional outlier datasets, custom calibration objectives, or model ensembling. Through empirical studies with different OOD detection settings -- far OOD, near OOD, and semantically coherent OOD - we establish a highly effective OOD detection approach. Code and models can be accessed here -- https://github.com/rushilanirudh/AMP

Related papers

• Model-free Test Time Adaptation for Out-Of-Distribution Detection [62.49795078366206]
  We propose a Non-Parametric Test Time textbfAdaptation framework for textbfDistribution textbfDetection (abbr) abbr utilizes online test samples for model adaptation during testing, enhancing adaptability to changing data distributions. We demonstrate the effectiveness of abbr through comprehensive experiments on multiple OOD detection benchmarks.
  arXiv  Detail & Related papers  (2023-11-28T02:00:47Z)
• Classifier-head Informed Feature Masking and Prototype-based Logit Smoothing for Out-of-Distribution Detection [27.062465089674763]
  Out-of-distribution (OOD) detection is essential when deploying neural networks in the real world. One main challenge is that neural networks often make overconfident predictions on OOD data. We propose an effective post-hoc OOD detection method based on a new feature masking strategy and a novel logit smoothing strategy.
  arXiv  Detail & Related papers  (2023-10-27T12:42:17Z)
• General-Purpose Multi-Modal OOD Detection Framework [5.287829685181842]
  Out-of-distribution (OOD) detection identifies test samples that differ from the training data, which is critical to ensuring the safety and reliability of machine learning (ML) systems. We propose a general-purpose weakly-supervised OOD detection framework, called WOOD, that combines a binary classifier and a contrastive learning component. We evaluate the proposed WOOD model on multiple real-world datasets, and the experimental results demonstrate that the WOOD model outperforms the state-of-the-art methods for multi-modal OOD detection.
  arXiv  Detail & Related papers  (2023-07-24T18:50:49Z)
• Out-of-distribution Detection with Implicit Outlier Transformation [72.73711947366377]
  Outlier exposure (OE) is powerful in out-of-distribution (OOD) detection. We propose a novel OE-based approach that makes the model perform well for unseen OOD situations.
  arXiv  Detail & Related papers  (2023-03-09T04:36:38Z)
• Unsupervised Evaluation of Out-of-distribution Detection: A Data-centric Perspective [55.45202687256175]
  Out-of-distribution (OOD) detection methods assume that they have test ground truths, i.e., whether individual test samples are in-distribution (IND) or OOD. In this paper, we are the first to introduce the unsupervised evaluation problem in OOD detection. We propose three methods to compute Gscore as an unsupervised indicator of OOD detection performance.
  arXiv  Detail & Related papers  (2023-02-16T13:34:35Z)
• On the Usefulness of Deep Ensemble Diversity for Out-of-Distribution Detection [7.221206118679026]
  The ability to detect Out-of-Distribution (OOD) data is important in safety-critical applications of deep learning. An existing intuition in the literature is that the diversity of Deep Ensemble predictions indicates distributional shift. We show experimentally that this intuition is not valid on ImageNet-scale OOD detection.
  arXiv  Detail & Related papers  (2022-07-15T15:02:38Z)
• WOOD: Wasserstein-based Out-of-Distribution Detection [6.163329453024915]
  Training data for deep-neural-network-based classifiers are usually assumed to be sampled from the same distribution. When part of the test samples are drawn from a distribution that is far away from that of the training samples, the trained neural network has a tendency to make high confidence predictions for these OOD samples. We propose a Wasserstein-based out-of-distribution detection (WOOD) method to overcome these challenges.
  arXiv  Detail & Related papers  (2021-12-13T02:35:15Z)
• Exploring Covariate and Concept Shift for Detection and Calibration of Out-of-Distribution Data [77.27338842609153]
  characterization reveals that sensitivity to each type of shift is important to the detection and confidence calibration of OOD data. We propose a geometrically-inspired method to improve OOD detection under both shifts with only in-distribution data. We are the first to propose a method that works well across both OOD detection and calibration and under different types of shifts.
  arXiv  Detail & Related papers  (2021-10-28T15:42:55Z)
• Robust Out-of-distribution Detection for Neural Networks [51.19164318924997]
  We show that existing detection mechanisms can be extremely brittle when evaluating on in-distribution and OOD inputs. We propose an effective algorithm called ALOE, which performs robust training by exposing the model to both adversarially crafted inlier and outlier examples.
  arXiv  Detail & Related papers  (2020-03-21T17:46:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.