Aleatoric and Epistemic Uncertainty with Random Forests

• URL: http://arxiv.org/abs/2001.00893v1
• Date: Fri, 3 Jan 2020 17:08:44 GMT
• Title: Aleatoric and Epistemic Uncertainty with Random Forests
• Authors: Mohammad Hossein Shaker and Eyke H\"ullermeier
• Abstract summary: We show how two approaches for measuring the learner's aleatoric and epistemic uncertainty in a prediction can be instantiated with decision trees and random forests. In this paper, we also compare random forests with deep neural networks, which have been used for a similar purpose.
• Score: 3.1410342959104725
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Due to the steadily increasing relevance of machine learning for practical applications, many of which are coming with safety requirements, the notion of uncertainty has received increasing attention in machine learning research in the last couple of years. In particular, the idea of distinguishing between two important types of uncertainty, often refereed to as aleatoric and epistemic, has recently been studied in the setting of supervised learning. In this paper, we propose to quantify these uncertainties with random forests. More specifically, we show how two general approaches for measuring the learner's aleatoric and epistemic uncertainty in a prediction can be instantiated with decision trees and random forests as learning algorithms in a classification setting. In this regard, we also compare random forests with deep neural networks, which have been used for a similar purpose.

Related papers

• One step closer to unbiased aleatoric uncertainty estimation [71.55174353766289]
  We propose a new estimation method by actively de-noising the observed data. By conducting a broad range of experiments, we demonstrate that our proposed approach provides a much closer approximation to the actual data uncertainty than the standard method.
  arXiv  Detail & Related papers  (2023-12-16T14:59:11Z)
• A Saliency-based Clustering Framework for Identifying Aberrant Predictions [49.1574468325115]
  We introduce the concept of aberrant predictions, emphasizing that the nature of classification errors is as critical as their frequency. We propose a novel, efficient training methodology aimed at both reducing the misclassification rate and discerning aberrant predictions. We apply this methodology to the less-explored domain of veterinary radiology, where the stakes are high but have not been as extensively studied compared to human medicine.
  arXiv  Detail & Related papers  (2023-11-11T01:53:59Z)
• Capsa: A Unified Framework for Quantifying Risk in Deep Neural Networks [142.67349734180445]
  Existing algorithms that provide risk-awareness to deep neural networks are complex and ad-hoc. Here we present capsa, a framework for extending models with risk-awareness.
  arXiv  Detail & Related papers  (2023-08-01T02:07:47Z)
• Sources of Uncertainty in Machine Learning -- A Statisticians' View [3.1498833540989413]
  The paper aims to formalize the two types of uncertainty associated with machine learning. Drawing parallels between statistical concepts and uncertainty in machine learning, we also demonstrate the role of data and their influence on uncertainty.
  arXiv  Detail & Related papers  (2023-05-26T07:44:19Z)
• On Second-Order Scoring Rules for Epistemic Uncertainty Quantification [8.298716599039501]
  We show that there seems to be no loss function that provides an incentive for a second-order learner to faithfully represent its uncertainty. As a main mathematical tool to prove this result, we introduce the generalised notion of second-order scoring rules.
  arXiv  Detail & Related papers  (2023-01-30T08:59:45Z)
• Introduction and Exemplars of Uncertainty Decomposition [3.0349501539299686]
  Uncertainty plays a crucial role in the machine learning field. This report aims to demystify the notion of uncertainty decomposition through an introduction to two types of uncertainty and several decomposition exemplars.
  arXiv  Detail & Related papers  (2022-11-17T17:14:34Z)
• The Unreasonable Effectiveness of Deep Evidential Regression [72.30888739450343]
  A new approach with uncertainty-aware regression-based neural networks (NNs) shows promise over traditional deterministic methods and typical Bayesian NNs. We detail the theoretical shortcomings and analyze the performance on synthetic and real-world data sets, showing that Deep Evidential Regression is a quantification rather than an exact uncertainty.
  arXiv  Detail & Related papers  (2022-05-20T10:10:32Z)
• Ensemble-based Uncertainty Quantification: Bayesian versus Credal Inference [0.0]
  We consider ensemble-based approaches to uncertainty quantification. We specifically focus on Bayesian methods and approaches based on so-called credal sets. The effectiveness of corresponding measures is evaluated and compared in an empirical study on classification with a reject option.
  arXiv  Detail & Related papers  (2021-07-21T22:47:24Z)
• Multivariate Deep Evidential Regression [77.34726150561087]
  A new approach with uncertainty-aware neural networks shows promise over traditional deterministic methods. We discuss three issues with a proposed solution to extract aleatoric and epistemic uncertainties from regression-based neural networks.
  arXiv  Detail & Related papers  (2021-04-13T12:20:18Z)
• Towards Robust Classification with Deep Generative Forests [13.096855747795303]
  Decision Trees and Random Forests are among the most widely used machine learning models. Being primarily discriminative models they lack principled methods to manipulate the uncertainty of predictions. We exploit Generative Forests (GeFs) to extend Random Forests to generative models representing the full joint distribution over the feature space.
  arXiv  Detail & Related papers  (2020-07-11T08:57:52Z)
• Hidden Cost of Randomized Smoothing [72.93630656906599]
  In this paper, we point out the side effects of current randomized smoothing. Specifically, we articulate and prove two major points: 1) the decision boundaries of smoothed classifiers will shrink, resulting in disparity in class-wise accuracy; 2) applying noise augmentation in the training process does not necessarily resolve the shrinking issue due to the inconsistent learning objectives.
  arXiv  Detail & Related papers  (2020-03-02T23:37:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.