Hierarchical VAEs Know What They Don't Know
- URL: http://arxiv.org/abs/2102.08248v1
- Date: Tue, 16 Feb 2021 16:08:04 GMT
- Title: Hierarchical VAEs Know What They Don't Know
- Authors: Jakob D. Havtorn, Jes Frellsen, S{\o}ren Hauberg, Lars Maal{\o}e
- Abstract summary: We develop a fast, scalable and fully unsupervised likelihood-ratio score for OOD detection.
We achieve state-of-the-art results on out-of-distribution detection.
- Score: 6.649455007186671
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Deep generative models have shown themselves to be state-of-the-art density
estimators. Yet, recent work has found that they often assign a higher
likelihood to data from outside the training distribution. This seemingly
paradoxical behavior has caused concerns over the quality of the attained
density estimates. In the context of hierarchical variational autoencoders, we
provide evidence to explain this behavior by out-of-distribution data having
in-distribution low-level features. We argue that this is both expected and
desirable behavior. With this insight in hand, we develop a fast, scalable and
fully unsupervised likelihood-ratio score for OOD detection that requires data
to be in-distribution across all feature-levels. We benchmark the method on a
vast set of data and model combinations and achieve state-of-the-art results on
out-of-distribution detection.
Related papers
- EAT: Towards Long-Tailed Out-of-Distribution Detection [55.380390767978554]
This paper addresses the challenging task of long-tailed OOD detection.
The main difficulty lies in distinguishing OOD data from samples belonging to the tail classes.
We propose two simple ideas: (1) Expanding the in-distribution class space by introducing multiple abstention classes, and (2) Augmenting the context-limited tail classes by overlaying images onto the context-rich OOD data.
arXiv Detail & Related papers (2023-12-14T13:47:13Z) - Augmenting Softmax Information for Selective Classification with
Out-of-Distribution Data [7.221206118679026]
We show that existing post-hoc methods perform quite differently compared to when evaluated only on OOD detection.
We propose a novel method for SCOD, Softmax Information Retaining Combination (SIRC), that augments softmax-based confidence scores with feature-agnostic information.
Experiments on a wide variety of ImageNet-scale datasets and convolutional neural network architectures show that SIRC is able to consistently match or outperform the baseline for SCOD.
arXiv Detail & Related papers (2022-07-15T14:39:57Z) - Leveraging Unlabeled Data to Predict Out-of-Distribution Performance [63.740181251997306]
Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions.
In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data.
We propose Average Thresholded Confidence (ATC), a practical method that learns a threshold on the model's confidence, predicting accuracy as the fraction of unlabeled examples.
arXiv Detail & Related papers (2022-01-11T23:01:12Z) - Understanding Failures in Out-of-Distribution Detection with Deep
Generative Models [22.11487118547924]
We prove that no method can guarantee performance beyond random chance without assumptions on which out-distributions are relevant.
We highlight the consequences implied by assuming support overlap between in- and out-distributions.
Our results suggest that estimation error is a more plausible explanation than the misalignment between likelihood-based OOD detection and out-distributions of interest.
arXiv Detail & Related papers (2021-07-14T18:00:11Z) - Robust Out-of-Distribution Detection on Deep Probabilistic Generative
Models [0.06372261626436676]
Out-of-distribution (OOD) detection is an important task in machine learning systems.
Deep probabilistic generative models facilitate OOD detection by estimating the likelihood of a data sample.
We propose a new detection metric that operates without outlier exposure.
arXiv Detail & Related papers (2021-06-15T06:36:10Z) - The Hidden Uncertainty in a Neural Networks Activations [105.4223982696279]
The distribution of a neural network's latent representations has been successfully used to detect out-of-distribution (OOD) data.
This work investigates whether this distribution correlates with a model's epistemic uncertainty, thus indicating its ability to generalise to novel inputs.
arXiv Detail & Related papers (2020-12-05T17:30:35Z) - Unlabelled Data Improves Bayesian Uncertainty Calibration under
Covariate Shift [100.52588638477862]
We develop an approximate Bayesian inference scheme based on posterior regularisation.
We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
arXiv Detail & Related papers (2020-06-26T13:50:19Z) - TraDE: Transformers for Density Estimation [101.20137732920718]
TraDE is a self-attention-based architecture for auto-regressive density estimation.
We present a suite of tasks such as regression using generated samples, out-of-distribution detection, and robustness to noise in the training data.
arXiv Detail & Related papers (2020-04-06T07:32:51Z) - Diversity inducing Information Bottleneck in Model Ensembles [73.80615604822435]
In this paper, we target the problem of generating effective ensembles of neural networks by encouraging diversity in prediction.
We explicitly optimize a diversity inducing adversarial loss for learning latent variables and thereby obtain diversity in the output predictions necessary for modeling multi-modal data.
Compared to the most competitive baselines, we show significant improvements in classification accuracy, under a shift in the data distribution.
arXiv Detail & Related papers (2020-03-10T03:10:41Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.