Rethinking Aleatoric and Epistemic Uncertainty

• URL: http://arxiv.org/abs/2412.20892v1
• Date: Mon, 30 Dec 2024 12:04:36 GMT
• Title: Rethinking Aleatoric and Epistemic Uncertainty
• Authors: Freddie Bickford Smith, Jannik Kossen, Eleanor Trollope, Mark van der Wilk, Adam Foster, Tom Rainforth,
• Abstract summary: We argue that the aleatoric-epistemic view is insufficiently expressive to capture all of the distinct quantities that researchers are interested in. We derive a simple delineation of different model-based uncertainties and the data-generating processes associated with training and evaluation.
• Score: 27.424543269616386
• License:
• Abstract: The ideas of aleatoric and epistemic uncertainty are widely used to reason about the probabilistic predictions of machine-learning models. We identify incoherence in existing discussions of these ideas and suggest this stems from the aleatoric-epistemic view being insufficiently expressive to capture all of the distinct quantities that researchers are interested in. To explain and address this we derive a simple delineation of different model-based uncertainties and the data-generating processes associated with training and evaluation. Using this in place of the aleatoric-epistemic view could produce clearer discourse as the field moves forward.

Related papers

• The Curious Case of Arbitrariness in Machine Learning [4.932130498861987]
  Algorithmic modelling relies on limited information in data to extrapolate outcomes for unseen scenarios, often embedding an element of arbitrariness in its decisions. A perspective on this arbitrariness that has recently gained interest is multiplicity-the study of arbitrariness across a set of "good models" We systemize the literature on multiplicity by: (a) formalizing the terminology around model design choices and their contribution to arbitrariness, (b) expanding the definition of multiplicity to incorporate underrepresented forms beyond just predictions and explanations, and (d) distilling the benefits and potential risks of multiplicity
  arXiv  Detail & Related papers  (2025-01-24T22:45:09Z)
• Seeing Unseen: Discover Novel Biomedical Concepts via Geometry-Constrained Probabilistic Modeling [53.7117640028211]
  We present a geometry-constrained probabilistic modeling treatment to resolve the identified issues. We incorporate a suite of critical geometric properties to impose proper constraints on the layout of constructed embedding space. A spectral graph-theoretic method is devised to estimate the number of potential novel classes.
  arXiv  Detail & Related papers  (2024-03-02T00:56:05Z)
• Invariant Probabilistic Prediction [45.90606906307022]
  We show that arbitrary distribution shifts do not, in general, admit invariant and robust probabilistic predictions. We propose a method to yield invariant probabilistic predictions, called IPP, and study the consistency of the underlying parameters.
  arXiv  Detail & Related papers  (2023-09-18T18:50:24Z)
• Quantification of Predictive Uncertainty via Inference-Time Sampling [57.749601811982096]
  We propose a post-hoc sampling strategy for estimating predictive uncertainty accounting for data ambiguity. The method can generate different plausible outputs for a given input and does not assume parametric forms of predictive distributions.
  arXiv  Detail & Related papers  (2023-08-03T12:43:21Z)
• Invariant Causal Set Covering Machines [64.86459157191346]
  Rule-based models, such as decision trees, appeal to practitioners due to their interpretable nature. However, the learning algorithms that produce such models are often vulnerable to spurious associations and thus, they are not guaranteed to extract causally-relevant insights. We propose Invariant Causal Set Covering Machines, an extension of the classical Set Covering Machine algorithm for conjunctions/disjunctions of binary-valued rules that provably avoids spurious associations.
  arXiv  Detail & Related papers  (2023-06-07T20:52:01Z)
• Uncertainty-Aware Pedestrian Trajectory Prediction via Distributional Diffusion [26.715578412088327]
  We present a model-agnostic uncertainty-aware pedestrian trajectory prediction framework. Unlike previous studies, we translate the predictiveity to explicit distributions, allowing it to generate plausible future trajectories. Our framework is compatible with different neural net architectures.
  arXiv  Detail & Related papers  (2023-03-15T04:58:43Z)
• Certifying Fairness of Probabilistic Circuits [33.1089249944851]
  We propose an algorithm to search for discrimination patterns in a general class of probabilistic models, namely probabilistic circuits. We also introduce new classes of patterns such as minimal, maximal, and optimal patterns that can effectively summarize exponentially many discrimination patterns.
  arXiv  Detail & Related papers  (2022-12-05T18:36:45Z)
• Uncertain Evidence in Probabilistic Models and Stochastic Simulators [80.40110074847527]
  We consider the problem of performing Bayesian inference in probabilistic models where observations are accompanied by uncertainty, referred to as uncertain evidence' We explore how to interpret uncertain evidence, and by extension the importance of proper interpretation as it pertains to inference about latent variables. We devise concrete guidelines on how to account for uncertain evidence and we provide new insights, particularly regarding consistency.
  arXiv  Detail & Related papers  (2022-10-21T20:32:59Z)
• Epistemic Deep Learning [6.144873990390373]
  We introduce a concept called epistemic deep learning based on the random-set interpretation of belief functions. We propose a novel random-set convolutional neural network for classification that produces scores for sets of classes by learning set-valued ground truth representations.
  arXiv  Detail & Related papers  (2022-06-15T15:39:52Z)
• The Unreasonable Effectiveness of Deep Evidential Regression [72.30888739450343]
  A new approach with uncertainty-aware regression-based neural networks (NNs) shows promise over traditional deterministic methods and typical Bayesian NNs. We detail the theoretical shortcomings and analyze the performance on synthetic and real-world data sets, showing that Deep Evidential Regression is a quantification rather than an exact uncertainty.
  arXiv  Detail & Related papers  (2022-05-20T10:10:32Z)
• Dense Uncertainty Estimation via an Ensemble-based Conditional Latent Variable Model [68.34559610536614]
  We argue that the aleatoric uncertainty is an inherent attribute of the data and can only be correctly estimated with an unbiased oracle model. We propose a new sampling and selection strategy at train time to approximate the oracle model for aleatoric uncertainty estimation. Our results show that our solution achieves both accurate deterministic results and reliable uncertainty estimation.
  arXiv  Detail & Related papers  (2021-11-22T08:54:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.