AUTO: Adaptive Outlier Optimization for Test-Time OOD Detection

• URL: http://arxiv.org/abs/2303.12267v2
• Date: Sat, 17 May 2025 08:05:59 GMT
• Title: AUTO: Adaptive Outlier Optimization for Test-Time OOD Detection
• Authors: Puning Yang, Jian Liang, Jie Cao, Ran He,
• Abstract summary: Out-of-distribution (OOD) detection aims to detect test samples that do not fall into any training in-distribution (ID) classes.<n>Data safety and privacy make it infeasible to collect task-specific outliers in advance for different scenarios.<n>We present test-time OOD detection, which allows the deployed model to utilize real OOD data from the unlabeled data stream during testing.
• Score: 79.51071170042972
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Out-of-distribution (OOD) detection aims to detect test samples that do not fall into any training in-distribution (ID) classes. Prior efforts focus on regularizing models with ID data only, largely underperforming counterparts that utilize auxiliary outliers. However, data safety and privacy make it infeasible to collect task-specific outliers in advance for different scenarios. Besides, using task-irrelevant outliers leads to inferior OOD detection performance. To address the above issue, we present a new setup called test-time OOD detection, which allows the deployed model to utilize real OOD data from the unlabeled data stream during testing. We propose Adaptive Outlier Optimization (AUTO) which allows for continuous adaptation of the OOD detector. Specifically, AUTO consists of three key components: 1) an in-out-aware filter to selectively annotate test samples with pseudo-ID and pseudo-OOD and ingeniously trigger the updating process while encountering each pseudo-OOD sample; 2) a dynamic-updated memory to overcome the catastrophic forgetting led by frequent parameter updates; 3) a prediction-aligning objective to calibrate the rough OOD objective during testing. Extensive experiments show that AUTO significantly improves OOD detection performance over state-of-the-art methods. Besides, evaluations on complicated scenarios (e.g. multi-OOD, time-series OOD) also conduct the superiority of AUTO.

Related papers

• OODD: Test-time Out-of-Distribution Detection with Dynamic Dictionary [10.19006278808466]
  Out-of-distribution (OOD) detection remains challenging for deep learning models. We propose OODD, a novel test-time OOD detection method that dynamically maintains and updates an OOD dictionary without fine-tuning. We present an optimized variant of the KNN-based OOD detection framework that achieves a 3x speedup while maintaining detection performance.
  arXiv  Detail & Related papers  (2025-03-13T15:41:56Z)
• phepy: Visual Benchmarks and Improvements for Out-of-Distribution Detectors [0.0]
  Testing OOD detection methods on real-world datasets is complicated by the ambiguity around which inputs are in-distribution (ID) or OOD.<n>We design a benchmark for OOD detection, which includes three novel and easily-visualisable toy examples.<n>We introduce two improvements, $t$-poking and OOD sample weighting, to make supervised detectors more precise at the ID-OOD boundary.
  arXiv  Detail & Related papers  (2025-03-07T06:25:20Z)
• Self-Calibrated Tuning of Vision-Language Models for Out-of-Distribution Detection [24.557227100200215]
  Out-of-distribution (OOD) detection is crucial for deploying reliable machine learning models in open-world applications. Recent advances in CLIP-based OOD detection have shown promising results via regularizing prompt tuning with OOD features extracted from ID data. We propose a novel framework, namely, Self-Calibrated Tuning (SCT), to mitigate this problem for effective OOD detection with only the given few-shot ID data.
  arXiv  Detail & Related papers  (2024-11-05T02:29:16Z)
• Enhancing OOD Detection Using Latent Diffusion [5.093257685701887]
  Out-of-Distribution (OOD) detection algorithms have been developed to identify unknown samples or objects in real-world deployments. We propose an Outlier Aware Learning framework, which synthesizes OOD training data in the latent space. We also develop a knowledge distillation module to prevent the degradation of ID classification accuracy when training with OOD data.
  arXiv  Detail & Related papers  (2024-06-24T11:01:43Z)
• Model-free Test Time Adaptation for Out-Of-Distribution Detection [62.49795078366206]
  We propose a Non-Parametric Test Time textbfAdaptation framework for textbfDistribution textbfDetection (abbr) abbr utilizes online test samples for model adaptation during testing, enhancing adaptability to changing data distributions. We demonstrate the effectiveness of abbr through comprehensive experiments on multiple OOD detection benchmarks.
  arXiv  Detail & Related papers  (2023-11-28T02:00:47Z)
• Can Pre-trained Networks Detect Familiar Out-of-Distribution Data? [37.36999826208225]
  We study the effect of PT-OOD on the OOD detection performance of pre-trained networks. We find that the low linear separability of PT-OOD in the feature space heavily degrades the PT-OOD detection performance. We propose a unique solution to large-scale pre-trained models: Leveraging powerful instance-by-instance discriminative representations of pre-trained models.
  arXiv  Detail & Related papers  (2023-10-02T02:01:00Z)
• SR-OOD: Out-of-Distribution Detection via Sample Repairing [48.272537939227206]
  Out-of-distribution (OOD) detection is a crucial task for ensuring the reliability and robustness of machine learning models. Recent works have shown that generative models often assign high confidence scores to OOD samples, indicating that they fail to capture the semantic information of the data. We take advantage of sample repairing and propose a novel OOD detection framework, namely SR-OOD. Our framework achieves superior performance over the state-of-the-art generative methods in OOD detection.
  arXiv  Detail & Related papers  (2023-05-26T16:35:20Z)
• Is Fine-tuning Needed? Pre-trained Language Models Are Near Perfect for Out-of-Domain Detection [28.810524375810736]
  Out-of-distribution (OOD) detection is a critical task for reliable predictions over text. Fine-tuning with pre-trained language models has been a de facto procedure to derive OOD detectors. We show that using distance-based detection methods, pre-trained language models are near-perfect OOD detectors when the distribution shift involves a domain change.
  arXiv  Detail & Related papers  (2023-05-22T17:42:44Z)
• Out-of-distribution Detection with Implicit Outlier Transformation [72.73711947366377]
  Outlier exposure (OE) is powerful in out-of-distribution (OOD) detection. We propose a novel OE-based approach that makes the model perform well for unseen OOD situations.
  arXiv  Detail & Related papers  (2023-03-09T04:36:38Z)
• Towards Realistic Out-of-Distribution Detection: A Novel Evaluation Framework for Improving Generalization in OOD Detection [14.541761912174799]
  This paper presents a novel evaluation framework for Out-of-Distribution (OOD) detection. It aims to assess the performance of machine learning models in more realistic settings.
  arXiv  Detail & Related papers  (2022-11-20T07:30:15Z)
• Provably Robust Detection of Out-of-distribution Data (almost) for free [124.14121487542613]
  Deep neural networks are known to produce highly overconfident predictions on out-of-distribution (OOD) data. In this paper we propose a novel method where from first principles we combine a certifiable OOD detector with a standard classifier into an OOD aware classifier. In this way we achieve the best of two worlds: certifiably adversarially robust OOD detection, even for OOD samples close to the in-distribution, without loss in prediction accuracy and close to state-of-the-art OOD detection performance for non-manipulated OOD data.
  arXiv  Detail & Related papers  (2021-06-08T11:40:49Z)
• Learn what you can't learn: Regularized Ensembles for Transductive Out-of-distribution Detection [76.39067237772286]
  We show that current out-of-distribution (OOD) detection algorithms for neural networks produce unsatisfactory results in a variety of OOD detection scenarios. This paper studies how such "hard" OOD scenarios can benefit from adjusting the detection method after observing a batch of the test data. We propose a novel method that uses an artificial labeling scheme for the test data and regularization to obtain ensembles of models that produce contradictory predictions only on the OOD samples in a test batch.
  arXiv  Detail & Related papers  (2020-12-10T16:55:13Z)
• ATOM: Robustifying Out-of-distribution Detection Using Outlier Mining [51.19164318924997]
  Adrial Training with informative Outlier Mining improves robustness of OOD detection. ATOM achieves state-of-the-art performance under a broad family of classic and adversarial OOD evaluation tasks.
  arXiv  Detail & Related papers  (2020-06-26T20:58:05Z)
• Robust Out-of-distribution Detection for Neural Networks [51.19164318924997]
  We show that existing detection mechanisms can be extremely brittle when evaluating on in-distribution and OOD inputs. We propose an effective algorithm called ALOE, which performs robust training by exposing the model to both adversarially crafted inlier and outlier examples.
  arXiv  Detail & Related papers  (2020-03-21T17:46:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.