A Critical Evaluation of Open-World Machine Learning

• URL: http://arxiv.org/abs/2007.04391v1
• Date: Wed, 8 Jul 2020 19:40:07 GMT
• Title: A Critical Evaluation of Open-World Machine Learning
• Authors: Liwei Song, Vikash Sehwag, Arjun Nitin Bhagoji, Prateek Mittal
• Abstract summary: Open-world machine learning (ML) combines closed-world models trained on in-distribution data with out-of-distribution (OOD) detectors. We show that the choice of in-distribution data, model architecture and OOD data have a strong impact on OOD detection performance. We show that OOD inputs with 22 unintentional corruptions or adversarial perturbations render open-world ML systems unusable with false positive rates of up to $100%$.
• Score: 46.88273149649151
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Open-world machine learning (ML) combines closed-world models trained on in-distribution data with out-of-distribution (OOD) detectors, which aim to detect and reject OOD inputs. Previous works on open-world ML systems usually fail to test their reliability under diverse, and possibly adversarial conditions. Therefore, in this paper, we seek to understand how resilient are state-of-the-art open-world ML systems to changes in system components? With our evaluation across 6 OOD detectors, we find that the choice of in-distribution data, model architecture and OOD data have a strong impact on OOD detection performance, inducing false positive rates in excess of $70\%$. We further show that OOD inputs with 22 unintentional corruptions or adversarial perturbations render open-world ML systems unusable with false positive rates of up to $100\%$. To increase the resilience of open-world ML, we combine robust classifiers with OOD detection techniques and uncover a new trade-off between OOD detection and robustness.

Related papers

• A Median Perspective on Unlabeled Data for Out-of-Distribution Detection [5.937613452723966]
  Out-of-distribution (OOD) detection plays a crucial role in ensuring the robustness and reliability of machine learning systems.<n>Recent approaches have explored the use of unlabeled data, showing potential for enhancing OOD detection capabilities.<n>We introduce Medix, a novel framework designed to identify potential outliers from unlabeled data using the median operation.
  arXiv  Detail & Related papers  (2025-10-07T22:43:57Z)
• Polysemantic Dropout: Conformal OOD Detection for Specialized LLMs [35.326974180503065]
  We propose a novel inference-time out-of-domain (OOD) detection algorithm for specialized large language models (LLMs)<n>Motivated by recent findings on polysemanticity and redundancy in LLMs, we hypothesize that in-domain inputs exhibit higher dropout tolerance than OOD inputs.<n>We aggregate dropout tolerance across multiple layers via a valid ensemble approach, improving detection while maintaining theoretical false alarm bounds from ICAD.
  arXiv  Detail & Related papers  (2025-09-04T20:50:51Z)
• Learning to Augment Distributions for Out-of-Distribution Detection [49.12437300327712]
  Open-world classification systems should discern out-of-distribution (OOD) data whose labels deviate from those of in-distribution (ID) cases. We propose Distributional-Augmented OOD Learning (DAL) to alleviating the OOD distribution discrepancy.
  arXiv  Detail & Related papers  (2023-11-03T09:19:33Z)
• Out-of-distribution Detection with Implicit Outlier Transformation [72.73711947366377]
  Outlier exposure (OE) is powerful in out-of-distribution (OOD) detection. We propose a novel OE-based approach that makes the model perform well for unseen OOD situations.
  arXiv  Detail & Related papers  (2023-03-09T04:36:38Z)
• Igeood: An Information Geometry Approach to Out-of-Distribution Detection [35.04325145919005]
  We introduce Igeood, an effective method for detecting out-of-distribution (OOD) samples. Igeood applies to any pre-trained neural network, works under various degrees of access to the machine learning model. We show that Igeood outperforms competing state-of-the-art methods on a variety of network architectures and datasets.
  arXiv  Detail & Related papers  (2022-03-15T11:26:35Z)
• Provably Robust Detection of Out-of-distribution Data (almost) for free [124.14121487542613]
  Deep neural networks are known to produce highly overconfident predictions on out-of-distribution (OOD) data. In this paper we propose a novel method where from first principles we combine a certifiable OOD detector with a standard classifier into an OOD aware classifier. In this way we achieve the best of two worlds: certifiably adversarially robust OOD detection, even for OOD samples close to the in-distribution, without loss in prediction accuracy and close to state-of-the-art OOD detection performance for non-manipulated OOD data.
  arXiv  Detail & Related papers  (2021-06-08T11:40:49Z)
• MOOD: Multi-level Out-of-distribution Detection [13.207044902083057]
  Out-of-distribution (OOD) detection is essential to prevent anomalous inputs from causing a model to fail during deployment. We propose a novel framework, multi-level out-of-distribution detection MOOD, which exploits intermediate classifier outputs for dynamic and efficient OOD inference. MOOD achieves up to 71.05% computational reduction in inference, while maintaining competitive OOD detection performance.
  arXiv  Detail & Related papers  (2021-04-30T02:18:31Z)
• ATOM: Robustifying Out-of-distribution Detection Using Outlier Mining [51.19164318924997]
  Adrial Training with informative Outlier Mining improves robustness of OOD detection. ATOM achieves state-of-the-art performance under a broad family of classic and adversarial OOD evaluation tasks.
  arXiv  Detail & Related papers  (2020-06-26T20:58:05Z)
• Robust Out-of-distribution Detection for Neural Networks [51.19164318924997]
  We show that existing detection mechanisms can be extremely brittle when evaluating on in-distribution and OOD inputs. We propose an effective algorithm called ALOE, which performs robust training by exposing the model to both adversarially crafted inlier and outlier examples.
  arXiv  Detail & Related papers  (2020-03-21T17:46:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.