DCV-ROOD Evaluation Framework: Dual Cross-Validation for Robust Out-of-Distribution Detection

• URL: http://arxiv.org/abs/2509.05778v1
• Date: Sat, 06 Sep 2025 17:20:09 GMT
• Title: DCV-ROOD Evaluation Framework: Dual Cross-Validation for Robust Out-of-Distribution Detection
• Authors: Arantxa Urrea-Castaño, Nicolás Segura-Kunsagi, Juan Luis Suárez-Díaz, Rosana Montes, Francisco Herrera,
• Abstract summary: Out-of-distribution (OOD) detection plays a key role in enhancing the robustness of artificial intelligence systems.<n>Cross-validation (CV) has proven to be a highly effective tool for providing a reasonable estimate of the performance of a learning algorithm.<n>This work proposes a dual CV framework for robust evaluation of OOD detection models.
• Score: 2.9726444682922897
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Out-of-distribution (OOD) detection plays a key role in enhancing the robustness of artificial intelligence systems by identifying inputs that differ significantly from the training distribution, thereby preventing unreliable predictions and enabling appropriate fallback mechanisms. Developing reliable OOD detection methods is a significant challenge, and rigorous evaluation of these techniques is essential for ensuring their effectiveness, as it allows researchers to assess their performance under diverse conditions and to identify potential limitations or failure modes. Cross-validation (CV) has proven to be a highly effective tool for providing a reasonable estimate of the performance of a learning algorithm. Although OOD scenarios exhibit particular characteristics, an appropriate adaptation of CV can lead to a suitable evaluation framework for this setting. This work proposes a dual CV framework for robust evaluation of OOD detection models, aimed at improving the reliability of their assessment. The proposed evaluation framework aims to effectively integrate in-distribution (ID) and OOD data while accounting for their differing characteristics. To achieve this, ID data are partitioned using a conventional approach, whereas OOD data are divided by grouping samples based on their classes. Furthermore, we analyze the context of data with class hierarchy to propose a data splitting that considers the entire class hierarchy to obtain fair ID-OOD partitions to apply the proposed evaluation framework. This framework is called Dual Cross-Validation for Robust Out-of-Distribution Detection (DCV-ROOD). To test the validity of the evaluation framework, we selected a set of state-of-the-art OOD detection methods, both with and without outlier exposure. The results show that the method achieves very fast convergence to the true performance.

Related papers

• Revisiting Logit Distributions for Reliable Out-of-Distribution Detection [73.9121001113687]
  Out-of-distribution (OOD) detection is critical for ensuring the reliability of deep learning models in open-world applications.<n>LogitGap is a novel post-hoc OOD detection method that exploits the relationship between the maximum logit and the remaining logits.<n>We show that LogitGap consistently achieves state-of-the-art performance across diverse OOD detection scenarios and benchmarks.
  arXiv  Detail & Related papers  (2025-10-23T02:16:45Z)
• Self-Calibrated Tuning of Vision-Language Models for Out-of-Distribution Detection [24.557227100200215]
  Out-of-distribution (OOD) detection is crucial for deploying reliable machine learning models in open-world applications. Recent advances in CLIP-based OOD detection have shown promising results via regularizing prompt tuning with OOD features extracted from ID data. We propose a novel framework, namely, Self-Calibrated Tuning (SCT), to mitigate this problem for effective OOD detection with only the given few-shot ID data.
  arXiv  Detail & Related papers  (2024-11-05T02:29:16Z)
• Margin-bounded Confidence Scores for Out-of-Distribution Detection [2.373572816573706]
  We propose a novel method called Margin bounded Confidence Scores (MaCS) to address the nontrivial OOD detection problem. MaCS enlarges the disparity between ID and OOD scores, which in turn makes the decision boundary more compact. Experiments on various benchmark datasets for image classification tasks demonstrate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2024-09-22T05:40:25Z)
• Model-free Test Time Adaptation for Out-Of-Distribution Detection [62.49795078366206]
  We propose a Non-Parametric Test Time textbfAdaptation framework for textbfDistribution textbfDetection (abbr) abbr utilizes online test samples for model adaptation during testing, enhancing adaptability to changing data distributions. We demonstrate the effectiveness of abbr through comprehensive experiments on multiple OOD detection benchmarks.
  arXiv  Detail & Related papers  (2023-11-28T02:00:47Z)
• Distilling the Unknown to Unveil Certainty [66.29929319664167]
  Out-of-distribution (OOD) detection is critical for identifying test samples that deviate from in-distribution (ID) data, ensuring network robustness and reliability.<n>This paper presents a flexible framework for OOD knowledge distillation that extracts OOD-sensitive information from a network to develop a binary classifier capable of distinguishing between ID and OOD samples.
  arXiv  Detail & Related papers  (2023-11-14T08:05:02Z)
• OOD Aware Supervised Contrastive Learning [13.329080722482187]
  Out-of-Distribution (OOD) detection is a crucial problem for the safe deployment of machine learning models. We leverage powerful representation learned with Supervised Contrastive (SupCon) training and propose a holistic approach to learn a robust to OOD data. Our solution is simple and efficient and acts as a natural extension of the closed-set supervised contrastive representation learning.
  arXiv  Detail & Related papers  (2023-10-03T10:38:39Z)
• Continual Evidential Deep Learning for Out-of-Distribution Detection [20.846788009755183]
  Uncertainty-based deep learning models have attracted a great deal of interest for their ability to provide accurate and reliable predictions. Evidential deep learning stands out achieving remarkable performance in detecting out-of-distribution (OOD) data with a single deterministic neural network. We propose the integration of an evidential deep learning method into a continual learning framework in order to perform simultaneously incremental object classification and OOD detection.
  arXiv  Detail & Related papers  (2023-09-06T13:36:59Z)
• Beyond AUROC & co. for evaluating out-of-distribution detection performance [50.88341818412508]
  Given their relevance for safe(r) AI, it is important to examine whether the basis for comparing OOD detection methods is consistent with practical needs. We propose a new metric - Area Under the Threshold Curve (AUTC), which explicitly penalizes poor separation between ID and OOD samples.
  arXiv  Detail & Related papers  (2023-06-26T12:51:32Z)
• AUTO: Adaptive Outlier Optimization for Test-Time OOD Detection [79.51071170042972]
  Out-of-distribution (OOD) detection aims to detect test samples that do not fall into any training in-distribution (ID) classes.<n>Data safety and privacy make it infeasible to collect task-specific outliers in advance for different scenarios.<n>We present test-time OOD detection, which allows the deployed model to utilize real OOD data from the unlabeled data stream during testing.
  arXiv  Detail & Related papers  (2023-03-22T02:28:54Z)
• Towards Realistic Out-of-Distribution Detection: A Novel Evaluation Framework for Improving Generalization in OOD Detection [14.541761912174799]
  This paper presents a novel evaluation framework for Out-of-Distribution (OOD) detection. It aims to assess the performance of machine learning models in more realistic settings.
  arXiv  Detail & Related papers  (2022-11-20T07:30:15Z)
• Provably Robust Detection of Out-of-distribution Data (almost) for free [124.14121487542613]
  Deep neural networks are known to produce highly overconfident predictions on out-of-distribution (OOD) data. In this paper we propose a novel method where from first principles we combine a certifiable OOD detector with a standard classifier into an OOD aware classifier. In this way we achieve the best of two worlds: certifiably adversarially robust OOD detection, even for OOD samples close to the in-distribution, without loss in prediction accuracy and close to state-of-the-art OOD detection performance for non-manipulated OOD data.
  arXiv  Detail & Related papers  (2021-06-08T11:40:49Z)
• Robust Out-of-distribution Detection for Neural Networks [51.19164318924997]
  We show that existing detection mechanisms can be extremely brittle when evaluating on in-distribution and OOD inputs. We propose an effective algorithm called ALOE, which performs robust training by exposing the model to both adversarially crafted inlier and outlier examples.
  arXiv  Detail & Related papers  (2020-03-21T17:46:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.