Deterministic Neural Networks with Appropriate Inductive Biases Capture Epistemic and Aleatoric Uncertainty

• URL: http://arxiv.org/abs/2102.11582v1
• Date: Tue, 23 Feb 2021 09:44:09 GMT
• Title: Deterministic Neural Networks with Appropriate Inductive Biases Capture Epistemic and Aleatoric Uncertainty
• Authors: Jishnu Mukhoti, Andreas Kirsch, Joost van Amersfoort, Philip H.S. Torr, Yarin Gal
• Abstract summary: We show that a single softmax neural net with minimal changes can beat the uncertainty predictions of Deep Ensembles. We study why, and show that with the right inductive biases, softmax neural nets trained with maximum likelihood reliably capture uncertainty through the feature-space density.
• Score: 91.01037972035635
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We show that a single softmax neural net with minimal changes can beat the uncertainty predictions of Deep Ensembles and other more complex single-forward-pass uncertainty approaches. Softmax neural nets cannot capture epistemic uncertainty reliably because for OoD points they extrapolate arbitrarily and suffer from feature collapse. This results in arbitrary softmax entropies for OoD points which can have high entropy, low, or anything in between. We study why, and show that with the right inductive biases, softmax neural nets trained with maximum likelihood reliably capture epistemic uncertainty through the feature-space density. This density is obtained using Gaussian Discriminant Analysis, but it cannot disentangle uncertainties. We show that it is necessary to combine this density with the softmax entropy to disentangle aleatoric and epistemic uncertainty -- crucial e.g. for active learning. We examine the quality of epistemic uncertainty on active learning and OoD detection, where we obtain SOTA ~0.98 AUROC on CIFAR-10 vs SVHN.

Related papers

• Density Uncertainty Layers for Reliable Uncertainty Estimation [20.867449366086237]
  Assessing the predictive uncertainty of deep neural networks is crucial for safety-related applications of deep learning. We propose a novel criterion for reliable predictive uncertainty: a model's predictive variance should be grounded in the empirical density of the input. Compared to existing approaches, density uncertainty layers provide more reliable uncertainty estimates and robust out-of-distribution detection performance.
  arXiv  Detail & Related papers  (2023-06-21T18:12:58Z)
• Beyond Voxel Prediction Uncertainty: Identifying brain lesions you can trust [1.1199585259018459]
  Deep neural networks have become the gold-standard approach for the automated segmentation of 3D medical images. In this work, we propose to go beyond voxel-wise assessment using an innovative Graph Neural Network approach. This network allows the fusion of three estimators of voxel uncertainty: entropy, variance, and model's confidence.
  arXiv  Detail & Related papers  (2022-09-22T09:20:05Z)
• The Unreasonable Effectiveness of Deep Evidential Regression [72.30888739450343]
  A new approach with uncertainty-aware regression-based neural networks (NNs) shows promise over traditional deterministic methods and typical Bayesian NNs. We detail the theoretical shortcomings and analyze the performance on synthetic and real-world data sets, showing that Deep Evidential Regression is a quantification rather than an exact uncertainty.
  arXiv  Detail & Related papers  (2022-05-20T10:10:32Z)
• A Deeper Look into Aleatoric and Epistemic Uncertainty Disentanglement [7.6146285961466]
  In this paper, we generalize methods to produce disentangled uncertainties to work with different uncertainty quantification methods. We show that there is an interaction between learning aleatoric and epistemic uncertainty, which is unexpected and violates assumptions on aleatoric uncertainty. We expect that our formulation and results help practitioners and researchers choose uncertainty methods and expand the use of disentangled uncertainties.
  arXiv  Detail & Related papers  (2022-04-20T08:41:37Z)
• Robust Depth Completion with Uncertainty-Driven Loss Functions [60.9237639890582]
  We introduce uncertainty-driven loss functions to improve the robustness of depth completion and handle the uncertainty in depth completion. Our method has been tested on KITTI Depth Completion Benchmark and achieved the state-of-the-art robustness performance in terms of MAE, IMAE, and IRMSE metrics.
  arXiv  Detail & Related papers  (2021-12-15T05:22:34Z)
• Understanding Softmax Confidence and Uncertainty [95.71801498763216]
  It is often remarked that neural networks fail to increase their uncertainty when predicting on data far from the training distribution. Yet naively using softmax confidence as a proxy for uncertainty achieves modest success in tasks exclusively testing for this. This paper investigates this contradiction, identifying two implicit biases that do encourage softmax confidence to correlate with uncertainty.
  arXiv  Detail & Related papers  (2021-06-09T10:37:29Z)
• The Hidden Uncertainty in a Neural Networks Activations [105.4223982696279]
  The distribution of a neural network's latent representations has been successfully used to detect out-of-distribution (OOD) data. This work investigates whether this distribution correlates with a model's epistemic uncertainty, thus indicating its ability to generalise to novel inputs.
  arXiv  Detail & Related papers  (2020-12-05T17:30:35Z)
• Localization Uncertainty Estimation for Anchor-Free Object Detection [48.931731695431374]
  There are several limitations of the existing uncertainty estimation methods for anchor-based object detection. We propose a new localization uncertainty estimation method called UAD for anchor-free object detection. Our method captures the uncertainty in four directions of box offsets that are homogeneous, so that it can tell which direction is uncertain.
  arXiv  Detail & Related papers  (2020-06-28T13:49:30Z)
• Entropic Out-of-Distribution Detection: Seamless Detection of Unknown Examples [8.284193221280214]
  We propose replacing SoftMax loss with a novel loss function that does not suffer from the mentioned weaknesses. The proposed IsoMax loss is isotropic (exclusively distance-based) and provides high entropy posterior probability distributions. Our experiments showed that IsoMax loss works as a seamless SoftMax loss drop-in replacement that significantly improves neural networks' OOD detection performance.
  arXiv  Detail & Related papers  (2020-06-07T00:34:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.