A Survey on Uncertainty Quantification Methods for Deep Learning

• URL: http://arxiv.org/abs/2302.13425v5
• Date: Mon, 15 Jul 2024 17:49:38 GMT
• Title: A Survey on Uncertainty Quantification Methods for Deep Learning
• Authors: Wenchong He, Zhe Jiang, Tingsong Xiao, Zelin Xu, Yukun Li,
• Abstract summary: Uncertainty quantification (UQ) aims to estimate the confidence of DNN predictions beyond prediction accuracy. This paper presents a systematic taxonomy of UQ methods for DNNs based on the types of uncertainty sources. We show how our taxonomy of UQ methodologies can potentially help guide the choice of UQ method in different machine learning problems.
• Score: 7.102893202197349
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Deep neural networks (DNNs) have achieved tremendous success in making accurate predictions for computer vision, natural language processing, as well as science and engineering domains. However, it is also well-recognized that DNNs sometimes make unexpected, incorrect, but overconfident predictions. This can cause serious consequences in high-stake applications, such as autonomous driving, medical diagnosis, and disaster response. Uncertainty quantification (UQ) aims to estimate the confidence of DNN predictions beyond prediction accuracy. In recent years, many UQ methods have been developed for DNNs. It is of great practical value to systematically categorize these UQ methods and compare their advantages and disadvantages. However, existing surveys mostly focus on categorizing UQ methodologies from a neural network architecture perspective or a Bayesian perspective and ignore the source of uncertainty that each methodology can incorporate, making it difficult to select an appropriate UQ method in practice. To fill the gap, this paper presents a systematic taxonomy of UQ methods for DNNs based on the types of uncertainty sources (data uncertainty versus model uncertainty). We summarize the advantages and disadvantages of methods in each category. We show how our taxonomy of UQ methodologies can potentially help guide the choice of UQ method in different machine learning problems (e.g., active learning, robustness, and reinforcement learning). We also identify current research gaps and propose several future research directions.

Related papers

• Uncertainty Quantification for Forward and Inverse Problems of PDEs via Latent Global Evolution [110.99891169486366]
  We propose a method that integrates efficient and precise uncertainty quantification into a deep learning-based surrogate model. Our method endows deep learning-based surrogate models with robust and efficient uncertainty quantification capabilities for both forward and inverse problems. Our method excels at propagating uncertainty over extended auto-regressive rollouts, making it suitable for scenarios involving long-term predictions.
  arXiv  Detail & Related papers  (2024-02-13T11:22:59Z)
• Online Algorithms with Uncertainty-Quantified Predictions [11.951228732915936]
  We investigate the problem of optimally utilizing uncertainty-quantified predictions in the design of online algorithms. In particular, we study two classic online problems, ski rental and online search, where the decision-maker is provided predictions augmented with UQ. We demonstrate that non-trivial modifications to algorithm design are needed to fully leverage the UQ predictions.
  arXiv  Detail & Related papers  (2023-10-17T20:09:41Z)
• Uncertainty in Natural Language Processing: Sources, Quantification, and Applications [56.130945359053776]
  We provide a comprehensive review of uncertainty-relevant works in the NLP field. We first categorize the sources of uncertainty in natural language into three types, including input, system, and output. We discuss the challenges of uncertainty estimation in NLP and discuss potential future directions.
  arXiv  Detail & Related papers  (2023-06-05T06:46:53Z)
• Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial [12.570694576213244]
  Uncertainty quantification (UQ) functions as an essential layer of safety assurance that could lead to more principled decision making. This tutorial provides a holistic lens on emerging UQ methods for ML models with a particular focus on neural networks. We discuss the increasingly important role of UQ of ML models in solving challenging problems in engineering design and health prognostics.
  arXiv  Detail & Related papers  (2023-05-07T03:12:03Z)
• Interpretable Self-Aware Neural Networks for Robust Trajectory Prediction [50.79827516897913]
  We introduce an interpretable paradigm for trajectory prediction that distributes the uncertainty among semantic concepts. We validate our approach on real-world autonomous driving data, demonstrating superior performance over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2022-11-16T06:28:20Z)
• Uncertainty Quantification in Neural Differential Equations [3.291910356217187]
  Uncertainty quantification (UQ) helps to make trustworthy predictions based on collected observations and uncertain domain knowledge. We adapt several state-of-the-art UQ methods to get the predictive uncertainty for DE solutions.
  arXiv  Detail & Related papers  (2021-11-08T00:09:39Z)
• MURAL: Meta-Learning Uncertainty-Aware Rewards for Outcome-Driven Reinforcement Learning [65.52675802289775]
  We show that an uncertainty aware classifier can solve challenging reinforcement learning problems. We propose a novel method for computing the normalized maximum likelihood (NML) distribution. We show that the resulting algorithm has a number of intriguing connections to both count-based exploration methods and prior algorithms for learning reward functions.
  arXiv  Detail & Related papers  (2021-07-15T08:19:57Z)
• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges [76.20963684020145]
  Uncertainty quantification (UQ) plays a pivotal role in reduction of uncertainties during both optimization and decision making processes. Bizarre approximation and ensemble learning techniques are two most widely-used UQ methods in the literature. This study reviews recent advances in UQ methods used in deep learning and investigates the application of these methods in reinforcement learning.
  arXiv  Detail & Related papers  (2020-11-12T06:41:05Z)
• Boosting Deep Neural Networks with Geometrical Prior Knowledge: A Survey [77.99182201815763]
  Deep Neural Networks (DNNs) achieve state-of-the-art results in many different problem settings. DNNs are often treated as black box systems, which complicates their evaluation and validation. One promising field, inspired by the success of convolutional neural networks (CNNs) in computer vision tasks, is to incorporate knowledge about symmetric geometrical transformations.
  arXiv  Detail & Related papers  (2020-06-30T14:56:05Z)
• Uncertainty Quantification Using Neural Networks for Molecular Property Prediction [33.34534208450156]
  We systematically evaluate several methods on five benchmark datasets using multiple complementary performance metrics. None of the methods we tested is unequivocally superior to all others, and none produces a particularly reliable ranking of errors across multiple datasets. We conclude with a practical recommendation as to which existing techniques seem to perform well relative to others.
  arXiv  Detail & Related papers  (2020-05-20T13:31:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.