Introduction and Exemplars of Uncertainty Decomposition

• URL: http://arxiv.org/abs/2211.15475v1
• Date: Thu, 17 Nov 2022 17:14:34 GMT
• Title: Introduction and Exemplars of Uncertainty Decomposition
• Authors: Shuo Chen
• Abstract summary: Uncertainty plays a crucial role in the machine learning field. This report aims to demystify the notion of uncertainty decomposition through an introduction to two types of uncertainty and several decomposition exemplars.
• Score: 3.0349501539299686
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Uncertainty plays a crucial role in the machine learning field. Both model trustworthiness and performance require the understanding of uncertainty, especially for models used in high-stake applications where errors can cause cataclysmic consequences, such as medical diagnosis and autonomous driving. Accordingly, uncertainty decomposition and quantification have attracted more and more attention in recent years. This short report aims to demystify the notion of uncertainty decomposition through an introduction to two types of uncertainty and several decomposition exemplars, including maximum likelihood estimation, Gaussian processes, deep neural network, and ensemble learning. In the end, cross connections to other topics in this seminar and two conclusions are provided.

Related papers

• A Review of Bayesian Uncertainty Quantification in Deep Probabilistic Image Segmentation [0.0]
  Advancements in image segmentation play an integral role within the greater scope of Deep Learning-based computer vision. Uncertainty quantification has been extensively studied within this context, enabling expression of model ignorance (epistemic uncertainty) or data ambiguity (aleatoric uncertainty) to prevent uninformed decision making. This work provides a comprehensive overview of probabilistic segmentation by discussing fundamental concepts in uncertainty that govern advancements in the field and the application to various tasks.
  arXiv  Detail & Related papers  (2024-11-25T13:26:09Z)
• Know Where You're Uncertain When Planning with Multimodal Foundation Models: A Formal Framework [54.40508478482667]
  We present a comprehensive framework to disentangle, quantify, and mitigate uncertainty in perception and plan generation. We propose methods tailored to the unique properties of perception and decision-making. We show that our uncertainty disentanglement framework reduces variability by up to 40% and enhances task success rates by 5% compared to baselines.
  arXiv  Detail & Related papers  (2024-11-03T17:32:00Z)
• Certainly Uncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awareness [106.52630978891054]
  We present a taxonomy of uncertainty specific to vision-language AI systems. We also introduce a new metric confidence-weighted accuracy, that is well correlated with both accuracy and calibration error.
  arXiv  Detail & Related papers  (2024-07-02T04:23:54Z)
• Is Epistemic Uncertainty Faithfully Represented by Evidential Deep Learning Methods? [26.344949402398917]
  This paper presents novel theoretical insights of evidential deep learning. It highlights the difficulties in optimizing second-order loss functions. It provides novel insights into issues of identifiability and convergence in second-order loss minimization.
  arXiv  Detail & Related papers  (2024-02-14T10:07:05Z)
• Nonparametric Identifiability of Causal Representations from Unknown Interventions [63.1354734978244]
  We study causal representation learning, the task of inferring latent causal variables and their causal relations from mixtures of the variables. Our goal is to identify both the ground truth latents and their causal graph up to a set of ambiguities which we show to be irresolvable from interventional data.
  arXiv  Detail & Related papers  (2023-06-01T10:51:58Z)
• Neural State-Space Models: Empirical Evaluation of Uncertainty Quantification [0.0]
  This paper presents preliminary results on uncertainty quantification for system identification with neural state-space models. We frame the learning problem in a Bayesian probabilistic setting and obtain posterior distributions for the neural network's weights and outputs. Based on the posterior, we construct credible intervals on the outputs and define a surprise index which can effectively diagnose usage of the model in a potentially dangerous out-of-distribution regime.
  arXiv  Detail & Related papers  (2023-04-13T08:57:33Z)
• Decision-Making Under Uncertainty: Beyond Probabilities [5.358161704743754]
  A classical assumption is that probabilities can sufficiently capture all uncertainty in a system. In this paper, the focus is on the uncertainty that goes beyond this classical interpretation. We show several solution techniques for both discrete and continuous models.
  arXiv  Detail & Related papers  (2023-03-10T10:53:33Z)
• The Unreasonable Effectiveness of Deep Evidential Regression [72.30888739450343]
  A new approach with uncertainty-aware regression-based neural networks (NNs) shows promise over traditional deterministic methods and typical Bayesian NNs. We detail the theoretical shortcomings and analyze the performance on synthetic and real-world data sets, showing that Deep Evidential Regression is a quantification rather than an exact uncertainty.
  arXiv  Detail & Related papers  (2022-05-20T10:10:32Z)
• A Survey on Epistemic (Model) Uncertainty in Supervised Learning: Recent Advances and Applications [18.731827159755014]
  Quantifying the uncertainty of supervised learning models plays an important role in making more reliable predictions. Epistemic uncertainty, which usually is due to insufficient knowledge about the model, can be reduced by collecting more data.
  arXiv  Detail & Related papers  (2021-11-03T01:22:46Z)
• BayesIMP: Uncertainty Quantification for Causal Data Fusion [52.184885680729224]
  We study the causal data fusion problem, where datasets pertaining to multiple causal graphs are combined to estimate the average treatment effect of a target variable. We introduce a framework which combines ideas from probabilistic integration and kernel mean embeddings to represent interventional distributions in the reproducing kernel Hilbert space.
  arXiv  Detail & Related papers  (2021-06-07T10:14:18Z)
• Discriminative Jackknife: Quantifying Uncertainty in Deep Learning via Higher-Order Influence Functions [121.10450359856242]
  We develop a frequentist procedure that utilizes influence functions of a model's loss functional to construct a jackknife (or leave-one-out) estimator of predictive confidence intervals. The DJ satisfies (1) and (2), is applicable to a wide range of deep learning models, is easy to implement, and can be applied in a post-hoc fashion without interfering with model training or compromising its accuracy.
  arXiv  Detail & Related papers  (2020-06-29T13:36:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.