Robust Out-of-Distribution Detection on Deep Probabilistic Generative Models

• URL: http://arxiv.org/abs/2106.07903v1
• Date: Tue, 15 Jun 2021 06:36:10 GMT
• Title: Robust Out-of-Distribution Detection on Deep Probabilistic Generative Models
• Authors: Jaemoo Choi, Changyeon Yoon, Jeongwoo Bae, Myungjoo Kang
• Abstract summary: Out-of-distribution (OOD) detection is an important task in machine learning systems. Deep probabilistic generative models facilitate OOD detection by estimating the likelihood of a data sample. We propose a new detection metric that operates without outlier exposure.
• Score: 0.06372261626436676
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Out-of-distribution (OOD) detection is an important task in machine learning systems for ensuring their reliability and safety. Deep probabilistic generative models facilitate OOD detection by estimating the likelihood of a data sample. However, such models frequently assign a suspiciously high likelihood to a specific outlier. Several recent works have addressed this issue by training a neural network with auxiliary outliers, which are generated by perturbing the input data. In this paper, we discover that these approaches fail for certain OOD datasets. Thus, we suggest a new detection metric that operates without outlier exposure. We observe that our metric is robust to diverse variations of an image compared to the previous outlier-exposing methods. Furthermore, our proposed score requires neither auxiliary models nor additional training. Instead, this paper utilizes the likelihood ratio statistic in a new perspective to extract genuine properties from the given single deep probabilistic generative model. We also apply a novel numerical approximation to enable fast implementation. Finally, we demonstrate comprehensive experiments on various probabilistic generative models and show that our method achieves state-of-the-art performance.

Related papers

• Forte : Finding Outliers with Representation Typicality Estimation [0.14061979259370275]
  Generative models can now produce synthetic data which is virtually indistinguishable from the real data used to train it. Recent work on OOD detection has raised doubts that generative model likelihoods are optimal OOD detectors. We introduce a novel approach that leverages representation learning, and informative summary statistics based on manifold estimation.
  arXiv  Detail & Related papers  (2024-10-02T08:26:37Z)
• Diversified Outlier Exposure for Out-of-Distribution Detection via Informative Extrapolation [110.34982764201689]
  Out-of-distribution (OOD) detection is important for deploying reliable machine learning models on real-world applications. Recent advances in outlier exposure have shown promising results on OOD detection via fine-tuning model with informatively sampled auxiliary outliers. We propose a novel framework, namely, Diversified Outlier Exposure (DivOE), for effective OOD detection via informative extrapolation based on the given auxiliary outliers.
  arXiv  Detail & Related papers  (2023-10-21T07:16:09Z)
• Unleashing Mask: Explore the Intrinsic Out-of-Distribution Detection Capability [70.72426887518517]
  Out-of-distribution (OOD) detection is an indispensable aspect of secure AI when deploying machine learning models in real-world applications. We propose a novel method, Unleashing Mask, which aims to restore the OOD discriminative capabilities of the well-trained model with ID data. Our method utilizes a mask to figure out the memorized atypical samples, and then finetune the model or prune it with the introduced mask to forget them.
  arXiv  Detail & Related papers  (2023-06-06T14:23:34Z)
• Watermarking for Out-of-distribution Detection [76.20630986010114]
  Out-of-distribution (OOD) detection aims to identify OOD data based on representations extracted from well-trained deep models. We propose a general methodology named watermarking in this paper. We learn a unified pattern that is superimposed onto features of original data, and the model's detection capability is largely boosted after watermarking.
  arXiv  Detail & Related papers  (2022-10-27T06:12:32Z)
• Reduced Robust Random Cut Forest for Out-Of-Distribution detection in machine learning models [0.799536002595393]
  Most machine learning-based regressors extract information from data collected via past observations of limited length to make predictions in the future. When input to these trained models is data with significantly different statistical properties from data used for training, there is no guarantee of accurate prediction. We introduce a novel approach for this detection process using a Reduced Robust Random Cut Forest data structure.
  arXiv  Detail & Related papers  (2022-06-18T17:01:40Z)
• Autoencoder Attractors for Uncertainty Estimation [13.618797548020462]
  We propose a novel approach for uncertainty estimation based on autoencoder models. We evaluate our approach on several dataset combinations as well as on an industrial application for occupant classification in the vehicle interior.
  arXiv  Detail & Related papers  (2022-04-01T12:10:06Z)
• Leveraging Unlabeled Data to Predict Out-of-Distribution Performance [63.740181251997306]
  Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions. In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data. We propose Average Thresholded Confidence (ATC), a practical method that learns a threshold on the model's confidence, predicting accuracy as the fraction of unlabeled examples.
  arXiv  Detail & Related papers  (2022-01-11T23:01:12Z)
• Sampling from Arbitrary Functions via PSD Models [55.41644538483948]
  We take a two-step approach by first modeling the probability distribution and then sampling from that model. We show that these models can approximate a large class of densities concisely using few evaluations, and present a simple algorithm to effectively sample from these models.
  arXiv  Detail & Related papers  (2021-10-20T12:25:22Z)
• Efficient remedies for outlier detection with variational autoencoders [8.80692072928023]
  Likelihoods computed by deep generative models are a candidate metric for outlier detection with unlabeled data. We show that a theoretically-grounded correction readily ameliorates a key bias with VAE likelihood estimates. We also show that the variance of the likelihoods computed over an ensemble of VAEs also enables robust outlier detection.
  arXiv  Detail & Related papers  (2021-08-19T16:00:58Z)
• Open Set Recognition with Conditional Probabilistic Generative Models [51.40872765917125]
  We propose Conditional Probabilistic Generative Models (CPGM) for open set recognition. CPGM can detect unknown samples but also classify known classes by forcing different latent features to approximate conditional Gaussian distributions. Experiment results on multiple benchmark datasets reveal that the proposed method significantly outperforms the baselines.
  arXiv  Detail & Related papers  (2020-08-12T06:23:49Z)
• Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder [6.767885381740952]
  probabilistic generative models can assign higher likelihoods on certain types of out-of-distribution samples. We propose Likelihood Regret, an efficient OOD score for VAEs.
  arXiv  Detail & Related papers  (2020-03-06T00:30:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.