Full-Spectrum Out-of-Distribution Detection

• URL: http://arxiv.org/abs/2204.05306v1
• Date: Mon, 11 Apr 2022 17:59:14 GMT
• Title: Full-Spectrum Out-of-Distribution Detection
• Authors: Jingkang Yang, Kaiyang Zhou, Ziwei Liu
• Abstract summary: We take into account both shift types and introduce full-spectrum OOD (FS-OOD) detection. We propose SEM, a simple feature-based semantics score function. SEM significantly outperforms current state-of-the-art methods.
• Score: 42.98617540431124
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing out-of-distribution (OOD) detection literature clearly defines semantic shift as a sign of OOD but does not have a consensus over covariate shift. Samples experiencing covariate shift but not semantic shift are either excluded from the test set or treated as OOD, which contradicts the primary goal in machine learning -- being able to generalize beyond the training distribution. In this paper, we take into account both shift types and introduce full-spectrum OOD (FS-OOD) detection, a more realistic problem setting that considers both detecting semantic shift and being tolerant to covariate shift; and designs three benchmarks. These new benchmarks have a more fine-grained categorization of distributions (i.e., training ID, covariate-shifted ID, near-OOD, and far-OOD) for the purpose of more comprehensively evaluating the pros and cons of algorithms. To address the FS-OOD detection problem, we propose SEM, a simple feature-based semantics score function. SEM is mainly composed of two probability measures: one is based on high-level features containing both semantic and non-semantic information, while the other is based on low-level feature statistics only capturing non-semantic image styles. With a simple combination, the non-semantic part is cancelled out, which leaves only semantic information in SEM that can better handle FS-OOD detection. Extensive experiments on the three new benchmarks show that SEM significantly outperforms current state-of-the-art methods. Our code and benchmarks are released in https://github.com/Jingkang50/OpenOOD.

Related papers

• Semantic or Covariate? A Study on the Intractable Case of Out-of-Distribution Detection [70.57120710151105]
  We provide a more precise definition of the Semantic Space for the ID distribution. We also define the "Tractable OOD" setting which ensures the distinguishability of OOD and ID distributions.
  arXiv  Detail & Related papers  (2024-11-18T03:09:39Z)
• What If the Input is Expanded in OOD Detection? [77.37433624869857]
  Out-of-distribution (OOD) detection aims to identify OOD inputs from unknown classes. Various scoring functions are proposed to distinguish it from in-distribution (ID) data. We introduce a novel perspective, i.e., employing different common corruptions on the input space.
  arXiv  Detail & Related papers  (2024-10-24T06:47:28Z)
• Margin-bounded Confidence Scores for Out-of-Distribution Detection [2.373572816573706]
  We propose a novel method called Margin bounded Confidence Scores (MaCS) to address the nontrivial OOD detection problem. MaCS enlarges the disparity between ID and OOD scores, which in turn makes the decision boundary more compact. Experiments on various benchmark datasets for image classification tasks demonstrate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2024-09-22T05:40:25Z)
• Rethinking the Evaluation of Out-of-Distribution Detection: A Sorites Paradox [70.57120710151105]
  Most existing out-of-distribution (OOD) detection benchmarks classify samples with novel labels as the OOD data. Some marginal OOD samples actually have close semantic contents to the in-distribution (ID) sample, which makes determining the OOD sample a Sorites Paradox. We construct a benchmark named Incremental Shift OOD (IS-OOD) to address the issue.
  arXiv  Detail & Related papers  (2024-06-14T09:27:56Z)
• Likelihood-Aware Semantic Alignment for Full-Spectrum Out-of-Distribution Detection [24.145060992747077]
  We propose a Likelihood-Aware Semantic Alignment (LSA) framework to promote the image-text correspondence into semantically high-likelihood regions. Extensive experiments demonstrate the remarkable OOD detection performance of our proposed LSA, surpassing existing methods by a margin of $15.26%$ and $18.88%$ on two F-OOD benchmarks.
  arXiv  Detail & Related papers  (2023-12-04T08:53:59Z)
• Out-of-Distributed Semantic Pruning for Robust Semi-Supervised Learning [17.409939628100517]
  We propose a unified framework termed OOD Semantic Pruning (OSP), which aims at pruning OOD semantics out from in-distribution (ID) features. OSP surpasses the previous state-of-the-art by 13.7% on accuracy for ID classification and 5.9% on AUROC for OOD detection on TinyImageNet dataset.
  arXiv  Detail & Related papers  (2023-05-29T15:37:07Z)
• Fine-Tuning Deteriorates General Textual Out-of-Distribution Detection by Distorting Task-Agnostic Features [14.325845491628087]
  Out-of-distribution (OOD) inputs are crucial for the safe deployment of natural language processing (NLP) models. We take the first step to evaluate the mainstream textual OOD detection methods for detecting semantic and non-semantic shifts. We present a simple yet effective general OOD score named GNOME that integrates the confidence scores derived from the task-agnostic and task-specific representations.
  arXiv  Detail & Related papers  (2023-01-30T08:01:13Z)
• Exploring Covariate and Concept Shift for Detection and Calibration of Out-of-Distribution Data [77.27338842609153]
  characterization reveals that sensitivity to each type of shift is important to the detection and confidence calibration of OOD data. We propose a geometrically-inspired method to improve OOD detection under both shifts with only in-distribution data. We are the first to propose a method that works well across both OOD detection and calibration and under different types of shifts.
  arXiv  Detail & Related papers  (2021-10-28T15:42:55Z)
• Triggering Failures: Out-Of-Distribution detection by learning from local adversarial attacks in Semantic Segmentation [76.2621758731288]
  We tackle the detection of out-of-distribution (OOD) objects in semantic segmentation. Our main contribution is a new OOD detection architecture called ObsNet associated with a dedicated training scheme based on Local Adversarial Attacks (LAA) We show it obtains top performances both in speed and accuracy when compared to ten recent methods of the literature on three different datasets.
  arXiv  Detail & Related papers  (2021-08-03T17:09:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.