A Mixture of Exemplars Approach for Efficient Out-of-Distribution Detection with Foundation Models

• URL: http://arxiv.org/abs/2311.17093v5
• Date: Sat, 08 Mar 2025 00:58:33 GMT
• Title: A Mixture of Exemplars Approach for Efficient Out-of-Distribution Detection with Foundation Models
• Authors: Evelyn Mannix, Howard Bondell,
• Abstract summary: This paper presents an efficient approach to tackling OOD detection that is designed to maximise the benefit of training with a high quality, frozen, pretrained foundation model.<n>MoLAR provides strong OOD performance when only comparing the similarity of OOD examples to the exemplars, a small set of images chosen to be representative of the dataset.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: One of the early weaknesses identified in deep neural networks trained for image classification tasks was their inability to provide low confidence predictions on out-of-distribution (OOD) data that was significantly different from the in-distribution (ID) data used to train them. Representation learning, where neural networks are trained in specific ways that improve their ability to detect OOD examples, has emerged as a promising solution. However, these approaches require long training times and can add additional overhead to detect OOD examples. Recent developments in Vision Transformer (ViT) foundation models$\unicode{x2013}$large networks trained on large and diverse datasets with self-supervised approaches$\unicode{x2013}$also show strong performance in OOD detection, and could address these challenges. This paper presents Mixture of Exemplars (MoLAR), an efficient approach to tackling OOD detection challenges that is designed to maximise the benefit of training a classifier with a high quality, frozen, pretrained foundation model backbone. MoLAR provides strong OOD performance when only comparing the similarity of OOD examples to the exemplars, a small set of images chosen to be representative of the dataset, leading to up to 30 times faster OOD detection inference over other methods that provide best performance when the full ID dataset is used. In some cases, only using these exemplars actually improves performance with MoLAR. Extensive experiments demonstrate the improved OOD detection performance of MoLAR in comparison to comparable approaches in both supervised and semi-supervised settings, and code is available at github.com/emannix/molar-mixture-of-exemplars.

Related papers

• WeiPer: OOD Detection using Weight Perturbations of Class Projections [11.130659240045544]
  We introduce perturbations of the class projections in the final fully connected layer which creates a richer representation of the input. We achieve state-of-the-art OOD detection results across multiple benchmarks of the OpenOOD framework.
  arXiv  Detail & Related papers  (2024-05-27T13:38:28Z)
• Gradient-Regularized Out-of-Distribution Detection [28.542499196417214]
  One of the challenges for neural networks in real-life applications is the overconfident errors these models make when the data is not from the original training distribution. We propose the idea of leveraging the information embedded in the gradient of the loss function during training to enable the network to learn a desired OOD score for each sample. We also develop a novel energy-based sampling method to allow the network to be exposed to more informative OOD samples during the training phase.
  arXiv  Detail & Related papers  (2024-04-18T17:50:23Z)
• EAT: Towards Long-Tailed Out-of-Distribution Detection [55.380390767978554]
  This paper addresses the challenging task of long-tailed OOD detection. The main difficulty lies in distinguishing OOD data from samples belonging to the tail classes. We propose two simple ideas: (1) Expanding the in-distribution class space by introducing multiple abstention classes, and (2) Augmenting the context-limited tail classes by overlaying images onto the context-rich OOD data.
  arXiv  Detail & Related papers  (2023-12-14T13:47:13Z)
• Can Pre-trained Networks Detect Familiar Out-of-Distribution Data? [37.36999826208225]
  We study the effect of PT-OOD on the OOD detection performance of pre-trained networks. We find that the low linear separability of PT-OOD in the feature space heavily degrades the PT-OOD detection performance. We propose a unique solution to large-scale pre-trained models: Leveraging powerful instance-by-instance discriminative representations of pre-trained models.
  arXiv  Detail & Related papers  (2023-10-02T02:01:00Z)
• From Global to Local: Multi-scale Out-of-distribution Detection [129.37607313927458]
  Out-of-distribution (OOD) detection aims to detect "unknown" data whose labels have not been seen during the in-distribution (ID) training process. Recent progress in representation learning gives rise to distance-based OOD detection. We propose Multi-scale OOD DEtection (MODE), a first framework leveraging both global visual information and local region details.
  arXiv  Detail & Related papers  (2023-08-20T11:56:25Z)
• Unleashing Mask: Explore the Intrinsic Out-of-Distribution Detection Capability [70.72426887518517]
  Out-of-distribution (OOD) detection is an indispensable aspect of secure AI when deploying machine learning models in real-world applications. We propose a novel method, Unleashing Mask, which aims to restore the OOD discriminative capabilities of the well-trained model with ID data. Our method utilizes a mask to figure out the memorized atypical samples, and then finetune the model or prune it with the introduced mask to forget them.
  arXiv  Detail & Related papers  (2023-06-06T14:23:34Z)
• AUTO: Adaptive Outlier Optimization for Online Test-Time OOD Detection [81.49353397201887]
  Out-of-distribution (OOD) detection is crucial to deploying machine learning models in open-world applications. We introduce a novel paradigm called test-time OOD detection, which utilizes unlabeled online data directly at test time to improve OOD detection performance. We propose adaptive outlier optimization (AUTO), which consists of an in-out-aware filter, an ID memory bank, and a semantically-consistent objective.
  arXiv  Detail & Related papers  (2023-03-22T02:28:54Z)
• Effective Robustness against Natural Distribution Shifts for Models with Different Training Data [113.21868839569]
  "Effective robustness" measures the extra out-of-distribution robustness beyond what can be predicted from the in-distribution (ID) performance. We propose a new evaluation metric to evaluate and compare the effective robustness of models trained on different data.
  arXiv  Detail & Related papers  (2023-02-02T19:28:41Z)
• A Simple Test-Time Method for Out-of-Distribution Detection [45.11199798139358]
  This paper proposes a simple Test-time Linear Training (ETLT) method for OOD detection. We find that the probabilities of input images being out-of-distribution are surprisingly linearly correlated to the features extracted by neural networks. We propose an online variant of the proposed method, which achieves promising performance and is more practical in real-world applications.
  arXiv  Detail & Related papers  (2022-07-17T16:02:58Z)
• No Shifted Augmentations (NSA): compact distributions for robust self-supervised Anomaly Detection [4.243926243206826]
  Unsupervised Anomaly detection (AD) requires building a notion of normalcy, distinguishing in-distribution (ID) and out-of-distribution (OOD) data. We investigate how the emph geometrical compactness of the ID feature distribution makes isolating and detecting outliers easier. We propose novel architectural modifications to the self-supervised feature learning step, that enable such compact distributions for ID data to be learned.
  arXiv  Detail & Related papers  (2022-03-19T15:55:32Z)
• No True State-of-the-Art? OOD Detection Methods are Inconsistent across Datasets [69.725266027309]
  Out-of-distribution detection is an important component of reliable ML systems. In this work, we show that none of these methods are inherently better at OOD detection than others on a standardized set of 16 pairs. We also show that a method outperforming another on a certain (ID, OOD) pair may not do so in a low-data regime.
  arXiv  Detail & Related papers  (2021-09-12T16:35:00Z)
• OODformer: Out-Of-Distribution Detection Transformer [15.17006322500865]
  In real-world safety-critical applications, it is important to be aware if a new data point is OOD. This paper proposes a first-of-its-kind OOD detection architecture named OODformer.
  arXiv  Detail & Related papers  (2021-07-19T15:46:38Z)
• Enhancing the Generalization for Intent Classification and Out-of-Domain Detection in SLU [70.44344060176952]
  Intent classification is a major task in spoken language understanding (SLU) Recent works have shown that using extra data and labels can improve the OOD detection performance. This paper proposes to train a model with only IND data while supporting both IND intent classification and OOD detection.
  arXiv  Detail & Related papers  (2021-06-28T08:27:38Z)
• EARLIN: Early Out-of-Distribution Detection for Resource-efficient Collaborative Inference [4.826988182025783]
  Collaborative inference enables resource-constrained edge devices to make inferences by uploading inputs to a server. While this setup works cost-effectively for successful inferences, it severely underperforms when the model faces input samples on which the model was not trained. We propose a novel lightweight OOD detection approach that mines important features from the shallow layers of a pretrained CNN model.
  arXiv  Detail & Related papers  (2021-06-25T18:43:23Z)
• Learn what you can't learn: Regularized Ensembles for Transductive Out-of-distribution Detection [76.39067237772286]
  We show that current out-of-distribution (OOD) detection algorithms for neural networks produce unsatisfactory results in a variety of OOD detection scenarios. This paper studies how such "hard" OOD scenarios can benefit from adjusting the detection method after observing a batch of the test data. We propose a novel method that uses an artificial labeling scheme for the test data and regularization to obtain ensembles of models that produce contradictory predictions only on the OOD samples in a test batch.
  arXiv  Detail & Related papers  (2020-12-10T16:55:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.