Density Uncertainty Layers for Reliable Uncertainty Estimation

• URL: http://arxiv.org/abs/2306.12497v2
• Date: Mon, 4 Mar 2024 19:29:04 GMT
• Title: Density Uncertainty Layers for Reliable Uncertainty Estimation
• Authors: Yookoon Park, David M. Blei
• Abstract summary: Assessing the predictive uncertainty of deep neural networks is crucial for safety-related applications of deep learning. We propose a novel criterion for reliable predictive uncertainty: a model's predictive variance should be grounded in the empirical density of the input. Compared to existing approaches, density uncertainty layers provide more reliable uncertainty estimates and robust out-of-distribution detection performance.
• Score: 20.867449366086237
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Assessing the predictive uncertainty of deep neural networks is crucial for safety-related applications of deep learning. Although Bayesian deep learning offers a principled framework for estimating model uncertainty, the common approaches that approximate the parameter posterior often fail to deliver reliable estimates of predictive uncertainty. In this paper, we propose a novel criterion for reliable predictive uncertainty: a model's predictive variance should be grounded in the empirical density of the input. That is, the model should produce higher uncertainty for inputs that are improbable in the training data and lower uncertainty for inputs that are more probable. To operationalize this criterion, we develop the density uncertainty layer, a stochastic neural network architecture that satisfies the density uncertain criterion by design. We study density uncertainty layers on the UCI and CIFAR-10/100 uncertainty benchmarks. Compared to existing approaches, density uncertainty layers provide more reliable uncertainty estimates and robust out-of-distribution detection performance.

Related papers

• One step closer to unbiased aleatoric uncertainty estimation [71.55174353766289]
  We propose a new estimation method by actively de-noising the observed data. By conducting a broad range of experiments, we demonstrate that our proposed approach provides a much closer approximation to the actual data uncertainty than the standard method.
  arXiv  Detail & Related papers  (2023-12-16T14:59:11Z)
• Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling [69.83976050879318]
  In large language models (LLMs), identifying sources of uncertainty is an important step toward improving reliability, trustworthiness, and interpretability. In this paper, we introduce an uncertainty decomposition framework for LLMs, called input clarification ensembling. Our approach generates a set of clarifications for the input, feeds them into an LLM, and ensembles the corresponding predictions.
  arXiv  Detail & Related papers  (2023-11-15T05:58:35Z)
• Adaptive Uncertainty Estimation via High-Dimensional Testing on Latent Representations [28.875819909902244]
  Uncertainty estimation aims to evaluate the confidence of a trained deep neural network. Existing uncertainty estimation approaches rely on low-dimensional distributional assumptions. We propose a new framework using data-adaptive high-dimensional hypothesis testing for uncertainty estimation.
  arXiv  Detail & Related papers  (2023-10-25T12:22:18Z)
• Integrating Uncertainty into Neural Network-based Speech Enhancement [27.868722093985006]
  Supervised masking approaches in the time-frequency domain aim to employ deep neural networks to estimate a multiplicative mask to extract clean speech. This leads to a single estimate for each input without any guarantees or measures of reliability. We study the benefits of modeling uncertainty in clean speech estimation.
  arXiv  Detail & Related papers  (2023-05-15T15:55:12Z)
• Reliability-Aware Prediction via Uncertainty Learning for Person Image Retrieval [51.83967175585896]
  UAL aims at providing reliability-aware predictions by considering data uncertainty and model uncertainty simultaneously. Data uncertainty captures the noise" inherent in the sample, while model uncertainty depicts the model's confidence in the sample's prediction.
  arXiv  Detail & Related papers  (2022-10-24T17:53:20Z)
• On Attacking Out-Domain Uncertainty Estimation in Deep Neural Networks [11.929914721626849]
  We show that state-of-the-art uncertainty estimation algorithms could fail catastrophically under our proposed adversarial attack. In particular, we aim at attacking the out-domain uncertainty estimation.
  arXiv  Detail & Related papers  (2022-10-03T23:33:38Z)
• Dense Uncertainty Estimation via an Ensemble-based Conditional Latent Variable Model [68.34559610536614]
  We argue that the aleatoric uncertainty is an inherent attribute of the data and can only be correctly estimated with an unbiased oracle model. We propose a new sampling and selection strategy at train time to approximate the oracle model for aleatoric uncertainty estimation. Our results show that our solution achieves both accurate deterministic results and reliable uncertainty estimation.
  arXiv  Detail & Related papers  (2021-11-22T08:54:10Z)
• Dense Uncertainty Estimation [62.23555922631451]
  In this paper, we investigate neural networks and uncertainty estimation techniques to achieve both accurate deterministic prediction and reliable uncertainty estimation. We work on two types of uncertainty estimations solutions, namely ensemble based methods and generative model based methods, and explain their pros and cons while using them in fully/semi/weakly-supervised framework.
  arXiv  Detail & Related papers  (2021-10-13T01:23:48Z)
• Approaching Neural Network Uncertainty Realism [53.308409014122816]
  Quantifying or at least upper-bounding uncertainties is vital for safety-critical systems such as autonomous vehicles. We evaluate uncertainty realism -- a strict quality criterion -- with a Mahalanobis distance-based statistical test. We adopt it to the automotive domain and show that it significantly improves uncertainty realism compared to a plain encoder-decoder model.
  arXiv  Detail & Related papers  (2021-01-08T11:56:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.