Perfect density models cannot guarantee anomaly detection

• URL: http://arxiv.org/abs/2012.03808v2
• Date: Tue, 23 Feb 2021 18:03:29 GMT
• Title: Perfect density models cannot guarantee anomaly detection
• Authors: Charline Le Lan, Laurent Dinh
• Abstract summary: In this paper, we take a closer look at the behavior of distribution densities and show that these quantities carry less meaningful information than previously thought. We conclude that the use of these likelihoods for out-of-distribution detection relies on strong and implicit hypotheses.
• Score: 9.64157020816848
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Thanks to the tractability of their likelihood, some deep generative models show promise for seemingly straightforward but important applications like anomaly detection, uncertainty estimation, and active learning. However, the likelihood values empirically attributed to anomalies conflict with the expectations these proposed applications suggest. In this paper, we take a closer look at the behavior of distribution densities and show that these quantities carry less meaningful information than previously thought, beyond estimation issues or the curse of dimensionality. We conclude that the use of these likelihoods for out-of-distribution detection relies on strong and implicit hypotheses, and highlight the necessity of explicitly formulating these assumptions for reliable anomaly detection.

Related papers

• Valid causal inference with unobserved confounding in high-dimensional settings [0.0]
  We show how valid semiparametric inference can be obtained in the presence of unobserved confounders and high-dimensional nuisance models. We propose uncertainty intervals which allow for unobserved confounding, and show that the resulting inference is valid when the amount of unobserved confounding is small.
  arXiv  Detail & Related papers  (2024-01-12T13:21:20Z)
• One step closer to unbiased aleatoric uncertainty estimation [71.55174353766289]
  We propose a new estimation method by actively de-noising the observed data. By conducting a broad range of experiments, we demonstrate that our proposed approach provides a much closer approximation to the actual data uncertainty than the standard method.
  arXiv  Detail & Related papers  (2023-12-16T14:59:11Z)
• A Data-Driven Measure of Relative Uncertainty for Misclassification Detection [25.947610541430013]
  We introduce a data-driven measure of uncertainty relative to an observer for misclassification detection. By learning patterns in the distribution of soft-predictions, our uncertainty measure can identify misclassified samples. We demonstrate empirical improvements over multiple image classification tasks, outperforming state-of-the-art misclassification detection methods.
  arXiv  Detail & Related papers  (2023-06-02T17:32:03Z)
• Monotonicity and Double Descent in Uncertainty Estimation with Gaussian Processes [52.92110730286403]
  It is commonly believed that the marginal likelihood should be reminiscent of cross-validation metrics and that both should deteriorate with larger input dimensions. We prove that by tuning hyper parameters, the performance, as measured by the marginal likelihood, improves monotonically with the input dimension. We also prove that cross-validation metrics exhibit qualitatively different behavior that is characteristic of double descent.
  arXiv  Detail & Related papers  (2022-10-14T08:09:33Z)
• Dense Uncertainty Estimation via an Ensemble-based Conditional Latent Variable Model [68.34559610536614]
  We argue that the aleatoric uncertainty is an inherent attribute of the data and can only be correctly estimated with an unbiased oracle model. We propose a new sampling and selection strategy at train time to approximate the oracle model for aleatoric uncertainty estimation. Our results show that our solution achieves both accurate deterministic results and reliable uncertainty estimation.
  arXiv  Detail & Related papers  (2021-11-22T08:54:10Z)
• Explainable Deep Few-shot Anomaly Detection with Deviation Networks [123.46611927225963]
  We introduce a novel weakly-supervised anomaly detection framework to train detection models. The proposed approach learns discriminative normality by leveraging the labeled anomalies and a prior probability. Our model is substantially more sample-efficient and robust, and performs significantly better than state-of-the-art competing methods in both closed-set and open-set settings.
  arXiv  Detail & Related papers  (2021-08-01T14:33:17Z)
• Understanding Failures in Out-of-Distribution Detection with Deep Generative Models [22.11487118547924]
  We prove that no method can guarantee performance beyond random chance without assumptions on which out-distributions are relevant. We highlight the consequences implied by assuming support overlap between in- and out-distributions. Our results suggest that estimation error is a more plausible explanation than the misalignment between likelihood-based OOD detection and out-distributions of interest.
  arXiv  Detail & Related papers  (2021-07-14T18:00:11Z)
• Deconfounded Score Method: Scoring DAGs with Dense Unobserved Confounding [101.35070661471124]
  We show that unobserved confounding leaves a characteristic footprint in the observed data distribution that allows for disentangling spurious and causal effects. We propose an adjusted score-based causal discovery algorithm that may be implemented with general-purpose solvers and scales to high-dimensional problems.
  arXiv  Detail & Related papers  (2021-03-28T11:07:59Z)
• The Hidden Uncertainty in a Neural Networks Activations [105.4223982696279]
  The distribution of a neural network's latent representations has been successfully used to detect out-of-distribution (OOD) data. This work investigates whether this distribution correlates with a model's epistemic uncertainty, thus indicating its ability to generalise to novel inputs.
  arXiv  Detail & Related papers  (2020-12-05T17:30:35Z)
• Further Analysis of Outlier Detection with Deep Generative Models [30.37180598197441]
  Deep generative models can frequently assign a higher likelihood to outliers. We present a possible explanation for this phenomenon, starting from the observation that a model's typical set and high-density region may not conincide. We also conduct additional experiments to help disentangle the impact of low-level texture versus high-level semantics in differentiating outliers.
  arXiv  Detail & Related papers  (2020-10-25T08:20:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.