Revisiting Deep Ensemble for Out-of-Distribution Detection: A Loss Landscape Perspective

• URL: http://arxiv.org/abs/2310.14227v2
• Date: Tue, 16 Jul 2024 01:49:19 GMT
• Title: Revisiting Deep Ensemble for Out-of-Distribution Detection: A Loss Landscape Perspective
• Authors: Kun Fang, Qinghua Tao, Xiaolin Huang, Jie Yang,
• Abstract summary: Out-of-Distribution (OoD) detection methods address to detect OoD samples from In-Distribution (InD) data. We propose a new perspective upon loss landscape and mode ensemble to investigate OoD detection.
• Score: 28.545919238058982
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing Out-of-Distribution (OoD) detection methods address to detect OoD samples from In-Distribution (InD) data mainly by exploring differences in features, logits and gradients in Deep Neural Networks (DNNs). We in this work propose a new perspective upon loss landscape and mode ensemble to investigate OoD detection. In the optimization of DNNs, there exist many local optima in the parameter space, or namely modes. Interestingly, we observe that these independent modes, which all reach low-loss regions with InD data (training and test data), yet yield significantly different loss landscapes with OoD data. Such an observation provides a novel view to investigate the OoD detection from the loss landscape, and further suggests significantly fluctuating OoD detection performance across these modes. For instance, FPR values of the RankFeat method can range from 46.58% to 84.70% among 5 modes, showing uncertain detection performance evaluations across independent modes. Motivated by such diversities on OoD loss landscape across modes, we revisit the deep ensemble method for OoD detection through mode ensemble, leading to improved performance and benefiting the OoD detector with reduced variances. Extensive experiments covering varied OoD detectors and network structures illustrate high variances across modes and validate the superiority of mode ensemble in boosting OoD detection. We hope this work could attract attention in the view of independent modes in the loss landscape of OoD data and more reliable evaluations on OoD detectors.

Related papers

• Beyond Perceptual Distances: Rethinking Disparity Assessment for Out-of-Distribution Detection with Diffusion Models [28.96695036746856]
  Out-of-Distribution (OoD) detection aims to justify whether a given sample is from the training distribution of the classifier-under-protection. DM-based methods bring fresh insights to the field, yet remain under-explored. Our work has demonstrated state-of-the-art detection performances among DM-based methods in extensive experiments.
  arXiv  Detail & Related papers  (2024-09-16T08:50:47Z)
• On the Robustness of Fully-Spiking Neural Networks in Open-World Scenarios using Forward-Only Learning Algorithms [6.7236795813629]
  We develop a novel algorithm for Out-of-Distribution (OoD) detection using the Forward-Forward Algorithm (FFA) Our approach measures the likelihood of a sample belonging to the in-distribution (ID) data by using the distance from the latent representation of samples to class-representative manifold. We also propose a gradient-free attribution technique that highlights the features of a sample pushing it away from the distribution of any class.
  arXiv  Detail & Related papers  (2024-07-19T08:08:17Z)
• Interpretable Out-Of-Distribution Detection Using Pattern Identification [0.0]
  Out-of-distribution (OoD) detection for data-based programs is a goal of paramount importance. Common approaches in the literature tend to train detectors requiring inside-of-distribution (in-distribution, or IoD) and OoD validation samples. We propose to use existing work from the field of explainable AI, namely the PARTICUL pattern identification algorithm, in order to build more interpretable and robust OoD detectors.
  arXiv  Detail & Related papers  (2023-01-24T15:35:54Z)
• The role of noise in denoising models for anomaly detection in medical images [62.0532151156057]
  Pathological brain lesions exhibit diverse appearance in brain images. Unsupervised anomaly detection approaches have been proposed using only normal data for training. We show that optimization of the spatial resolution and magnitude of the noise improves the performance of different model training regimes.
  arXiv  Detail & Related papers  (2023-01-19T21:39:38Z)
• Raising the Bar on the Evaluation of Out-of-Distribution Detection [88.70479625837152]
  We define 2 categories of OoD data using the subtly different concepts of perceptual/visual and semantic similarity to in-distribution (iD) data. We propose a GAN based framework for generating OoD samples from each of these 2 categories, given an iD dataset. We show that a) state-of-the-art OoD detection methods which perform exceedingly well on conventional benchmarks are significantly less robust to our proposed benchmark.
  arXiv  Detail & Related papers  (2022-09-24T08:48:36Z)
• Benchmarking Deep Models for Salient Object Detection [67.07247772280212]
  We construct a general SALient Object Detection (SALOD) benchmark to conduct a comprehensive comparison among several representative SOD methods. In the above experiments, we find that existing loss functions usually specialized in some metrics but reported inferior results on the others. We propose a novel Edge-Aware (EA) loss that promotes deep networks to learn more discriminative features by integrating both pixel- and image-level supervision signals.
  arXiv  Detail & Related papers  (2022-02-07T03:43:16Z)
• Entropy Maximization and Meta Classification for Out-Of-Distribution Detection in Semantic Segmentation [7.305019142196585]
  "Out-of-distribution" (OoD) samples are crucial for many applications such as automated driving. A natural baseline approach to OoD detection is to threshold on the pixel-wise softmax entropy. We present a two-step procedure that significantly improves that approach.
  arXiv  Detail & Related papers  (2020-12-09T11:01:06Z)
• NADS: Neural Architecture Distribution Search for Uncertainty Awareness [79.18710225716791]
  Machine learning (ML) systems often encounter Out-of-Distribution (OoD) errors when dealing with testing data coming from a distribution different from training data. Existing OoD detection approaches are prone to errors and even sometimes assign higher likelihoods to OoD samples. We propose Neural Architecture Distribution Search (NADS) to identify common building blocks among all uncertainty-aware architectures.
  arXiv  Detail & Related papers  (2020-06-11T17:39:07Z)
• Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data [87.61504710345528]
  We propose two strategies for freeing a neural network from tuning with OoD data, while improving its OoD detection performance. We specifically propose to decompose confidence scoring as well as a modified input pre-processing method. Our further analysis on a larger scale image dataset shows that the two types of distribution shifts, specifically semantic shift and non-semantic shift, present a significant difference.
  arXiv  Detail & Related papers  (2020-02-26T04:18:25Z)
• Simple and Effective Prevention of Mode Collapse in Deep One-Class Classification [93.2334223970488]
  We propose two regularizers to prevent hypersphere collapse in deep SVDD. The first regularizer is based on injecting random noise via the standard cross-entropy loss. The second regularizer penalizes the minibatch variance when it becomes too small.
  arXiv  Detail & Related papers  (2020-01-24T03:44:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.