Double Descent Meets Out-of-Distribution Detection: Theoretical Insights and Empirical Analysis on the role of model complexity

• URL: http://arxiv.org/abs/2411.02184v2
• Date: Tue, 27 May 2025 09:44:58 GMT
• Title: Double Descent Meets Out-of-Distribution Detection: Theoretical Insights and Empirical Analysis on the role of model complexity
• Authors: Mouïn Ben Ammar, David Brellmann, Arturo Mendoza, Antoine Manzanera, Gianni Franchi,
• Abstract summary: Out-of-distribution (OOD) detection is essential for ensuring the reliability and safety of machine learning systems.<n>In this paper, we focus on post-hoc OOD detection, which enables identifying OOD samples without altering the model's training procedure or objective.
• Score: 2.206582444513284
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Out-of-distribution (OOD) detection is essential for ensuring the reliability and safety of machine learning systems. In recent years, it has received increasing attention, particularly through post-hoc detection and training-based methods. In this paper, we focus on post-hoc OOD detection, which enables identifying OOD samples without altering the model's training procedure or objective. Our primary goal is to investigate the relationship between model capacity and its OOD detection performance. Specifically, we aim to answer the following question: Does the Double Descent phenomenon manifest in post-hoc OOD detection? This question is crucial, as it can reveal whether overparameterization, which is already known to benefit generalization, can also enhance OOD detection. Despite the growing interest in these topics by the classic supervised machine learning community, this intersection remains unexplored for OOD detection. We empirically demonstrate that the Double Descent effect does indeed appear in post-hoc OOD detection. Furthermore, we provide theoretical insights to explain why this phenomenon emerges in such setting. Finally, we show that the overparameterized regime does not yield superior results consistently, and we propose a method to identify the optimal regime for OOD detection based on our observations.

Related papers

• Self-Calibrated Tuning of Vision-Language Models for Out-of-Distribution Detection [24.557227100200215]
  Out-of-distribution (OOD) detection is crucial for deploying reliable machine learning models in open-world applications. Recent advances in CLIP-based OOD detection have shown promising results via regularizing prompt tuning with OOD features extracted from ID data. We propose a novel framework, namely, Self-Calibrated Tuning (SCT), to mitigate this problem for effective OOD detection with only the given few-shot ID data.
  arXiv  Detail & Related papers  (2024-11-05T02:29:16Z)
• The Best of Both Worlds: On the Dilemma of Out-of-distribution Detection [75.65876949930258]
  Out-of-distribution (OOD) detection is essential for model trustworthiness. We show that the superior OOD detection performance of state-of-the-art methods is achieved by secretly sacrificing the OOD generalization ability.
  arXiv  Detail & Related papers  (2024-10-12T07:02:04Z)
• Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Survey [107.08019135783444]
  Out-of-distribution (OOD) samples are crucial for ensuring the safety of machine learning systems.<n>Several other problems are closely related to OOD detection, including anomaly detection (AD), novelty detection (ND), open set recognition (OSR), and outlier detection (OD)
  arXiv  Detail & Related papers  (2024-07-31T17:59:58Z)
• Rethinking Out-of-Distribution Detection on Imbalanced Data Distribution [38.844580833635725]
  We present a training-time regularization technique to mitigate the bias and boost imbalanced OOD detectors across architecture designs. Our method translates into consistent improvements on the representative CIFAR10-LT, CIFAR100-LT, and ImageNet-LT benchmarks.
  arXiv  Detail & Related papers  (2024-07-23T12:28:59Z)
• Iterative Deployment Exposure for Unsupervised Out-of-Distribution Detection [5.019613806273252]
  Iterative Deployment Exposure (IDE) is a novel and more realistic setting for out-of-distribution (OOD) detection.<n> CSO uses a new U-OOD scoring function that combines the Mahalanobis distance with a nearest-neighbor approach.<n>We validate our approach on a dedicated benchmark, showing that our method greatly improves upon strong baselines on three medical imaging modalities.
  arXiv  Detail & Related papers  (2024-06-04T13:57:34Z)
• Optimizing OOD Detection in Molecular Graphs: A Novel Approach with Diffusion Models [71.39421638547164]
  We propose to detect OOD molecules by adopting an auxiliary diffusion model-based framework, which compares similarities between input molecules and reconstructed graphs. Due to the generative bias towards reconstructing ID training samples, the similarity scores of OOD molecules will be much lower to facilitate detection. Our research pioneers an approach of Prototypical Graph Reconstruction for Molecular OOD Detection, dubbed as PGR-MOOD and hinges on three innovations.
  arXiv  Detail & Related papers  (2024-04-24T03:25:53Z)
• Out-of-Distribution Data: An Acquaintance of Adversarial Examples -- A Survey [7.891552999555933]
  Deep neural networks (DNNs) deployed in real-world applications can encounter out-of-distribution (OOD) data and adversarial examples. Traditionally, research has addressed OOD detection and adversarial robustness as separate challenges. This survey focuses on the intersection of these two areas, examining how the research community has investigated them together.
  arXiv  Detail & Related papers  (2024-04-08T06:27:38Z)
• Detecting Out-of-Distribution Through the Lens of Neural Collapse [7.04686607977352]
  Out-of-distribution (OOD) detection is essential for safe deployment. Existing detectors exhibit generalization discrepancies and cost concerns. We propose a highly versatile and efficient OOD detector inspired by the trend of Neural Collapse.
  arXiv  Detail & Related papers  (2023-11-02T05:18:28Z)
• NECO: NEural Collapse Based Out-of-distribution detection [2.4958897155282282]
  We introduce NECO, a novel post-hoc method for OOD detection. Our experiments demonstrate that NECO achieves both small and large-scale OOD detection tasks. We provide a theoretical explanation for the effectiveness of our method in OOD detection.
  arXiv  Detail & Related papers  (2023-10-10T17:53:36Z)
• OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detection [82.85303878718207]
  Out-of-Distribution (OOD) detection is critical for the reliable operation of open-world intelligent systems.<n>This paper presents OpenOOD v1.5, a significant improvement from its predecessor that ensures accurate and standardized evaluation of OOD detection methodologies.
  arXiv  Detail & Related papers  (2023-06-15T17:28:00Z)
• LINe: Out-of-Distribution Detection by Leveraging Important Neurons [15.797257361788812]
  We introduce a new aspect for analyzing the difference in model outputs between in-distribution data and OOD data. We propose a novel method, Leveraging Important Neurons (LINe), for post-hoc Out of distribution detection.
  arXiv  Detail & Related papers  (2023-03-24T13:49:05Z)
• Rethinking Out-of-distribution (OOD) Detection: Masked Image Modeling is All You Need [52.88953913542445]
  We find surprisingly that simply using reconstruction-based methods could boost the performance of OOD detection significantly. We take Masked Image Modeling as a pretext task for our OOD detection framework (MOOD)
  arXiv  Detail & Related papers  (2023-02-06T08:24:41Z)
• Generalized Out-of-Distribution Detection: A Survey [83.0449593806175]
  Out-of-distribution (OOD) detection is critical to ensuring the reliability and safety of machine learning systems. Several other problems, including anomaly detection (AD), novelty detection (ND), open set recognition (OSR), and outlier detection (OD) are closely related to OOD detection. We first present a unified framework called generalized OOD detection, which encompasses the five aforementioned problems.
  arXiv  Detail & Related papers  (2021-10-21T17:59:41Z)
• On the Impact of Spurious Correlation for Out-of-distribution Detection [14.186776881154127]
  We present a new formalization and model the data shifts by taking into account both the invariant and environmental features. Our results suggest that the detection performance is severely worsened when the correlation between spurious features and labels is increased in the training set.
  arXiv  Detail & Related papers  (2021-09-12T23:58:17Z)
• Provably Robust Detection of Out-of-distribution Data (almost) for free [124.14121487542613]
  Deep neural networks are known to produce highly overconfident predictions on out-of-distribution (OOD) data. In this paper we propose a novel method where from first principles we combine a certifiable OOD detector with a standard classifier into an OOD aware classifier. In this way we achieve the best of two worlds: certifiably adversarially robust OOD detection, even for OOD samples close to the in-distribution, without loss in prediction accuracy and close to state-of-the-art OOD detection performance for non-manipulated OOD data.
  arXiv  Detail & Related papers  (2021-06-08T11:40:49Z)
• Label Smoothed Embedding Hypothesis for Out-of-Distribution Detection [72.35532598131176]
  We propose an unsupervised method to detect OOD samples using a $k$-NN density estimate. We leverage a recent insight about label smoothing, which we call the emphLabel Smoothed Embedding Hypothesis We show that our proposal outperforms many OOD baselines and also provide new finite-sample high-probability statistical results.
  arXiv  Detail & Related papers  (2021-02-09T21:04:44Z)
• Learn what you can't learn: Regularized Ensembles for Transductive Out-of-distribution Detection [76.39067237772286]
  We show that current out-of-distribution (OOD) detection algorithms for neural networks produce unsatisfactory results in a variety of OOD detection scenarios. This paper studies how such "hard" OOD scenarios can benefit from adjusting the detection method after observing a batch of the test data. We propose a novel method that uses an artificial labeling scheme for the test data and regularization to obtain ensembles of models that produce contradictory predictions only on the OOD samples in a test batch.
  arXiv  Detail & Related papers  (2020-12-10T16:55:13Z)
• Robust Out-of-distribution Detection for Neural Networks [51.19164318924997]
  We show that existing detection mechanisms can be extremely brittle when evaluating on in-distribution and OOD inputs. We propose an effective algorithm called ALOE, which performs robust training by exposing the model to both adversarially crafted inlier and outlier examples.
  arXiv  Detail & Related papers  (2020-03-21T17:46:28Z)
• Detecting Out-of-Distribution Examples with In-distribution Examples and Gram Matrices [8.611328447624679]
  Deep neural networks yield confident, incorrect predictions when presented with Out-of-Distribution examples. In this paper, we propose to detect OOD examples by identifying inconsistencies between activity patterns and class predicted. We find that characterizing activity patterns by Gram matrices and identifying anomalies in gram matrix values can yield high OOD detection rates.
  arXiv  Detail & Related papers  (2019-12-28T19:44:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.