Uncertainty-Aware Deep Classifiers using Generative Models
- URL: http://arxiv.org/abs/2006.04183v1
- Date: Sun, 7 Jun 2020 15:38:35 GMT
- Title: Uncertainty-Aware Deep Classifiers using Generative Models
- Authors: Murat Sensoy, Lance Kaplan, Federico Cerutti, Maryam Saleki
- Abstract summary: Deep neural networks are often ignorant about what they do not know and overconfident when they make uninformed predictions.
Some recent approaches quantify uncertainty directly by training the model to output high uncertainty for the data samples close to class boundaries or from the outside of the training distribution.
We develop a novel neural network model that is able to express both aleatoric and epistemic uncertainty to distinguish decision boundary and out-of-distribution regions.
- Score: 7.486679152591502
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Deep neural networks are often ignorant about what they do not know and
overconfident when they make uninformed predictions. Some recent approaches
quantify classification uncertainty directly by training the model to output
high uncertainty for the data samples close to class boundaries or from the
outside of the training distribution. These approaches use an auxiliary data
set during training to represent out-of-distribution samples. However,
selection or creation of such an auxiliary data set is non-trivial, especially
for high dimensional data such as images. In this work we develop a novel
neural network model that is able to express both aleatoric and epistemic
uncertainty to distinguish decision boundary and out-of-distribution regions of
the feature space. To this end, variational autoencoders and generative
adversarial networks are incorporated to automatically generate
out-of-distribution exemplars for training. Through extensive analysis, we
demonstrate that the proposed approach provides better estimates of uncertainty
for in- and out-of-distribution samples, and adversarial examples on well-known
data sets against state-of-the-art approaches including recent Bayesian
approaches for neural networks and anomaly detection methods.
Related papers
- Generative Posterior Networks for Approximately Bayesian Epistemic
Uncertainty Estimation [11.66240919177989]
We propose a new method, Generative Posterior Networks (GPNs), that uses unlabeled data to estimate uncertainty in high-dimensional problems.
A GPN is a generative model that, given a prior distribution over functions, approximates the posterior distribution directly by regularizing the network towards samples from the prior.
arXiv Detail & Related papers (2023-12-29T00:43:41Z) - Implicit Variational Inference for High-Dimensional Posteriors [7.924706533725115]
In variational inference, the benefits of Bayesian models rely on accurately capturing the true posterior distribution.
We propose using neural samplers that specify implicit distributions, which are well-suited for approximating complex multimodal and correlated posteriors.
Our approach introduces novel bounds for approximate inference using implicit distributions by locally linearising the neural sampler.
arXiv Detail & Related papers (2023-10-10T14:06:56Z) - Uncertainty Quantification in Deep Neural Networks through Statistical
Inference on Latent Space [0.0]
We develop an algorithm that exploits the latent-space representation of data points fed into the network to assess the accuracy of their prediction.
We show on a synthetic dataset that commonly used methods are mostly overconfident.
In contrast, our method can detect such out-of-distribution data points as inaccurately predicted, thus aiding in the automatic detection of outliers.
arXiv Detail & Related papers (2023-05-18T09:52:06Z) - NUQ: Nonparametric Uncertainty Quantification for Deterministic Neural
Networks [151.03112356092575]
We show the principled way to measure the uncertainty of predictions for a classifier based on Nadaraya-Watson's nonparametric estimate of the conditional label distribution.
We demonstrate the strong performance of the method in uncertainty estimation tasks on a variety of real-world image datasets.
arXiv Detail & Related papers (2022-02-07T12:30:45Z) - Training on Test Data with Bayesian Adaptation for Covariate Shift [96.3250517412545]
Deep neural networks often make inaccurate predictions with unreliable uncertainty estimates.
We derive a Bayesian model that provides for a well-defined relationship between unlabeled inputs under distributional shift and model parameters.
We show that our method improves both accuracy and uncertainty estimation.
arXiv Detail & Related papers (2021-09-27T01:09:08Z) - Improving Uncertainty Calibration via Prior Augmented Data [56.88185136509654]
Neural networks have proven successful at learning from complex data distributions by acting as universal function approximators.
They are often overconfident in their predictions, which leads to inaccurate and miscalibrated probabilistic predictions.
We propose a solution by seeking out regions of feature space where the model is unjustifiably overconfident, and conditionally raising the entropy of those predictions towards that of the prior distribution of the labels.
arXiv Detail & Related papers (2021-02-22T07:02:37Z) - Towards Trustworthy Predictions from Deep Neural Networks with Fast
Adversarial Calibration [2.8935588665357077]
We propose an efficient yet general modelling approach for obtaining well-calibrated, trustworthy probabilities for samples obtained after a domain shift.
We introduce a new training strategy combining an entropy-encouraging loss term with an adversarial calibration loss term and demonstrate that this results in well-calibrated and technically trustworthy predictions.
arXiv Detail & Related papers (2020-12-20T13:39:29Z) - Open Set Recognition with Conditional Probabilistic Generative Models [51.40872765917125]
We propose Conditional Probabilistic Generative Models (CPGM) for open set recognition.
CPGM can detect unknown samples but also classify known classes by forcing different latent features to approximate conditional Gaussian distributions.
Experiment results on multiple benchmark datasets reveal that the proposed method significantly outperforms the baselines.
arXiv Detail & Related papers (2020-08-12T06:23:49Z) - Unlabelled Data Improves Bayesian Uncertainty Calibration under
Covariate Shift [100.52588638477862]
We develop an approximate Bayesian inference scheme based on posterior regularisation.
We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
arXiv Detail & Related papers (2020-06-26T13:50:19Z) - Uncertainty Estimation Using a Single Deep Deterministic Neural Network [66.26231423824089]
We propose a method for training a deterministic deep model that can find and reject out of distribution data points at test time with a single forward pass.
We scale training in these with a novel loss function and centroid updating scheme and match the accuracy of softmax models.
arXiv Detail & Related papers (2020-03-04T12:27:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.