Generalized Out-of-Distribution Detection: A Survey
- URL: http://arxiv.org/abs/2110.11334v3
- Date: Tue, 23 Jan 2024 07:36:33 GMT
- Title: Generalized Out-of-Distribution Detection: A Survey
- Authors: Jingkang Yang, Kaiyang Zhou, Yixuan Li, Ziwei Liu
- Abstract summary: Out-of-distribution (OOD) detection is critical to ensuring the reliability and safety of machine learning systems.
Several other problems, including anomaly detection (AD), novelty detection (ND), open set recognition (OSR), and outlier detection (OD) are closely related to OOD detection.
We first present a unified framework called generalized OOD detection, which encompasses the five aforementioned problems.
- Score: 83.0449593806175
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Out-of-distribution (OOD) detection is critical to ensuring the reliability
and safety of machine learning systems. For instance, in autonomous driving, we
would like the driving system to issue an alert and hand over the control to
humans when it detects unusual scenes or objects that it has never seen during
training time and cannot make a safe decision. The term, OOD detection, first
emerged in 2017 and since then has received increasing attention from the
research community, leading to a plethora of methods developed, ranging from
classification-based to density-based to distance-based ones. Meanwhile,
several other problems, including anomaly detection (AD), novelty detection
(ND), open set recognition (OSR), and outlier detection (OD), are closely
related to OOD detection in terms of motivation and methodology. Despite common
goals, these topics develop in isolation, and their subtle differences in
definition and problem setting often confuse readers and practitioners. In this
survey, we first present a unified framework called generalized OOD detection,
which encompasses the five aforementioned problems, i.e., AD, ND, OSR, OOD
detection, and OD. Under our framework, these five problems can be seen as
special cases or sub-tasks, and are easier to distinguish. We then review each
of these five areas by summarizing their recent technical developments, with a
special focus on OOD detection methodologies. We conclude this survey with open
challenges and potential research directions.
Related papers
- The Best of Both Worlds: On the Dilemma of Out-of-distribution Detection [75.65876949930258]
Out-of-distribution (OOD) detection is essential for model trustworthiness.
We show that the superior OOD detection performance of state-of-the-art methods is achieved by secretly sacrificing the OOD generalization ability.
arXiv Detail & Related papers (2024-10-12T07:02:04Z) - Dissecting Out-of-Distribution Detection and Open-Set Recognition: A Critical Analysis of Methods and Benchmarks [17.520137576423593]
We aim to provide a consolidated view of the two largest sub-fields within the community: out-of-distribution (OOD) detection and open-set recognition (OSR)
We perform rigorous cross-evaluation between state-of-the-art methods in the OOD detection and OSR settings and identify a strong correlation between the performances of methods for them.
We propose a new, large-scale benchmark setting which we suggest better disentangles the problem tackled by OOD detection and OSR.
arXiv Detail & Related papers (2024-08-29T17:55:07Z) - Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Survey [107.08019135783444]
We first present a generalized OOD detection v2, encapsulating the evolution of AD, ND, OSR, OOD detection, and OD in the VLM era.
Our framework reveals that, with some field inactivity and integration, the demanding challenges have become OOD detection and AD.
arXiv Detail & Related papers (2024-07-31T17:59:58Z) - Rethinking Out-of-Distribution Detection for Reinforcement Learning: Advancing Methods for Evaluation and Detection [3.7384109981836158]
We study the problem of out-of-distribution (OOD) detection in reinforcement learning (RL)
We propose a clarification of terminology for OOD detection in RL, which aligns it with the literature from other machine learning domains.
We present new benchmark scenarios for OOD detection, which introduce anomalies with temporal autocorrelation into different components of the agent-environment loop.
We find that DEXTER can reliably identify anomalies across benchmark scenarios, exhibiting superior performance compared to both state-of-the-art OOD detectors and high-dimensional changepoint detectors adopted from statistics.
arXiv Detail & Related papers (2024-04-10T15:39:49Z) - Out-of-Distribution Data: An Acquaintance of Adversarial Examples -- A Survey [7.891552999555933]
Deep neural networks (DNNs) deployed in real-world applications can encounter out-of-distribution (OOD) data and adversarial examples.
Traditionally, research has addressed OOD detection and adversarial robustness as separate challenges.
This survey focuses on the intersection of these two areas, examining how the research community has investigated them together.
arXiv Detail & Related papers (2024-04-08T06:27:38Z) - Beyond AUROC & co. for evaluating out-of-distribution detection
performance [50.88341818412508]
Given their relevance for safe(r) AI, it is important to examine whether the basis for comparing OOD detection methods is consistent with practical needs.
We propose a new metric - Area Under the Threshold Curve (AUTC), which explicitly penalizes poor separation between ID and OOD samples.
arXiv Detail & Related papers (2023-06-26T12:51:32Z) - OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detection [82.85303878718207]
Out-of-Distribution (OOD) detection is critical for the reliable operation of open-world intelligent systems.
This paper presents OpenOOD v1.5, a significant improvement from its predecessor that ensures accurate, standardized, and user-friendly evaluation of OOD detection methodologies.
arXiv Detail & Related papers (2023-06-15T17:28:00Z) - OpenOOD: Benchmarking Generalized Out-of-Distribution Detection [60.13300701826931]
Out-of-distribution (OOD) detection is vital to safety-critical machine learning applications.
The field currently lacks a unified, strictly formulated, and comprehensive benchmark.
We build a unified, well-structured called OpenOOD, which implements over 30 methods developed in relevant fields.
arXiv Detail & Related papers (2022-10-13T17:59:57Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.