Double Descent Meets Out-of-Distribution Detection: Theoretical Insights and Empirical Analysis on the role of model complexity

• URL: http://arxiv.org/abs/2411.02184v1
• Date: Mon, 04 Nov 2024 15:39:12 GMT
• Title: Double Descent Meets Out-of-Distribution Detection: Theoretical Insights and Empirical Analysis on the role of model complexity
• Authors: Mouïn Ben Ammar, David Brellmann, Arturo Mendoza, Antoine Manzanera, Gianni Franchi,
• Abstract summary: We propose an expected OOD risk metric to evaluate classifiers confidence on both training and OOD samples. We show that the OOD risk depicts an infinite peak, when the number of parameters is equal to the number of samples.
• Score: 2.206582444513284
• License:
• Abstract: While overparameterization is known to benefit generalization, its impact on Out-Of-Distribution (OOD) detection is less understood. This paper investigates the influence of model complexity in OOD detection. We propose an expected OOD risk metric to evaluate classifiers confidence on both training and OOD samples. Leveraging Random Matrix Theory, we derive bounds for the expected OOD risk of binary least-squares classifiers applied to Gaussian data. We show that the OOD risk depicts an infinite peak, when the number of parameters is equal to the number of samples, which we associate with the double descent phenomenon. Our experimental study on different OOD detection methods across multiple neural architectures extends our theoretical insights and highlights a double descent curve. Our observations suggest that overparameterization does not necessarily lead to better OOD detection. Using the Neural Collapse framework, we provide insights to better understand this behavior. To facilitate reproducibility, our code will be made publicly available upon publication.

Related papers

• The Best of Both Worlds: On the Dilemma of Out-of-distribution Detection [75.65876949930258]
  Out-of-distribution (OOD) detection is essential for model trustworthiness. We show that the superior OOD detection performance of state-of-the-art methods is achieved by secretly sacrificing the OOD generalization ability.
  arXiv  Detail & Related papers  (2024-10-12T07:02:04Z)
• Rethinking Out-of-Distribution Detection on Imbalanced Data Distribution [38.844580833635725]
  We present a training-time regularization technique to mitigate the bias and boost imbalanced OOD detectors across architecture designs. Our method translates into consistent improvements on the representative CIFAR10-LT, CIFAR100-LT, and ImageNet-LT benchmarks.
  arXiv  Detail & Related papers  (2024-07-23T12:28:59Z)
• Optimizing OOD Detection in Molecular Graphs: A Novel Approach with Diffusion Models [71.39421638547164]
  We propose to detect OOD molecules by adopting an auxiliary diffusion model-based framework, which compares similarities between input molecules and reconstructed graphs. Due to the generative bias towards reconstructing ID training samples, the similarity scores of OOD molecules will be much lower to facilitate detection. Our research pioneers an approach of Prototypical Graph Reconstruction for Molecular OOD Detection, dubbed as PGR-MOOD and hinges on three innovations.
  arXiv  Detail & Related papers  (2024-04-24T03:25:53Z)
• Detecting Out-of-Distribution Through the Lens of Neural Collapse [7.04686607977352]
  Out-of-distribution (OOD) detection is essential for safe deployment. Existing detectors exhibit generalization discrepancies and cost concerns. We propose a highly versatile and efficient OOD detector inspired by the trend of Neural Collapse.
  arXiv  Detail & Related papers  (2023-11-02T05:18:28Z)
• NECO: NEural Collapse Based Out-of-distribution detection [2.4958897155282282]
  We introduce NECO, a novel post-hoc method for OOD detection. Our experiments demonstrate that NECO achieves both small and large-scale OOD detection tasks. We provide a theoretical explanation for the effectiveness of our method in OOD detection.
  arXiv  Detail & Related papers  (2023-10-10T17:53:36Z)
• LINe: Out-of-Distribution Detection by Leveraging Important Neurons [15.797257361788812]
  We introduce a new aspect for analyzing the difference in model outputs between in-distribution data and OOD data. We propose a novel method, Leveraging Important Neurons (LINe), for post-hoc Out of distribution detection.
  arXiv  Detail & Related papers  (2023-03-24T13:49:05Z)
• On the Impact of Spurious Correlation for Out-of-distribution Detection [14.186776881154127]
  We present a new formalization and model the data shifts by taking into account both the invariant and environmental features. Our results suggest that the detection performance is severely worsened when the correlation between spurious features and labels is increased in the training set.
  arXiv  Detail & Related papers  (2021-09-12T23:58:17Z)
• Provably Robust Detection of Out-of-distribution Data (almost) for free [124.14121487542613]
  Deep neural networks are known to produce highly overconfident predictions on out-of-distribution (OOD) data. In this paper we propose a novel method where from first principles we combine a certifiable OOD detector with a standard classifier into an OOD aware classifier. In this way we achieve the best of two worlds: certifiably adversarially robust OOD detection, even for OOD samples close to the in-distribution, without loss in prediction accuracy and close to state-of-the-art OOD detection performance for non-manipulated OOD data.
  arXiv  Detail & Related papers  (2021-06-08T11:40:49Z)
• Label Smoothed Embedding Hypothesis for Out-of-Distribution Detection [72.35532598131176]
  We propose an unsupervised method to detect OOD samples using a $k$-NN density estimate. We leverage a recent insight about label smoothing, which we call the emphLabel Smoothed Embedding Hypothesis We show that our proposal outperforms many OOD baselines and also provide new finite-sample high-probability statistical results.
  arXiv  Detail & Related papers  (2021-02-09T21:04:44Z)
• Learn what you can't learn: Regularized Ensembles for Transductive Out-of-distribution Detection [76.39067237772286]
  We show that current out-of-distribution (OOD) detection algorithms for neural networks produce unsatisfactory results in a variety of OOD detection scenarios. This paper studies how such "hard" OOD scenarios can benefit from adjusting the detection method after observing a batch of the test data. We propose a novel method that uses an artificial labeling scheme for the test data and regularization to obtain ensembles of models that produce contradictory predictions only on the OOD samples in a test batch.
  arXiv  Detail & Related papers  (2020-12-10T16:55:13Z)
• Robust Out-of-distribution Detection for Neural Networks [51.19164318924997]
  We show that existing detection mechanisms can be extremely brittle when evaluating on in-distribution and OOD inputs. We propose an effective algorithm called ALOE, which performs robust training by exposing the model to both adversarially crafted inlier and outlier examples.
  arXiv  Detail & Related papers  (2020-03-21T17:46:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.