Teaching Uncertainty Quantification in Machine Learning through Use Cases

• URL: http://arxiv.org/abs/2108.08712v1
• Date: Thu, 19 Aug 2021 14:22:17 GMT
• Title: Teaching Uncertainty Quantification in Machine Learning through Use Cases
• Authors: Matias Valdenegro-Toro
• Abstract summary: Uncertainty in machine learning is not generally taught as general knowledge in Machine Learning course curricula. We propose a short curriculum for a course about uncertainty in machine learning, and complement the course with a selection of use cases.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Uncertainty in machine learning is not generally taught as general knowledge in Machine Learning course curricula. In this paper we propose a short curriculum for a course about uncertainty in machine learning, and complement the course with a selection of use cases, aimed to trigger discussion and let students play with the concepts of uncertainty in a programming setting. Our use cases cover the concept of output uncertainty, Bayesian neural networks and weight distributions, sources of uncertainty, and out of distribution detection. We expect that this curriculum and set of use cases motivates the community to adopt these important concepts into courses for safety in AI.

Related papers

• Uncertainties of Latent Representations in Computer Vision [2.33877878310217]
  This thesis makes uncertainty estimates easily accessible by adding them to the latent representation vectors of pretrained computer vision models. We show that these unobservable uncertainties about unobservable latent representations are indeed provably correct.
  arXiv  Detail & Related papers  (2024-08-26T14:02:30Z)
• UnUnlearning: Unlearning is not sufficient for content regulation in advanced generative AI [50.61495097098296]
  We revisit the paradigm in which unlearning is used for Large Language Models (LLMs) We introduce a concept of ununlearning, where unlearned knowledge gets reintroduced in-context. We argue that content filtering for impermissible knowledge will be required and even exact unlearning schemes are not enough for effective content regulation.
  arXiv  Detail & Related papers  (2024-06-27T10:24:35Z)
• Brain-Inspired Computational Intelligence via Predictive Coding [89.6335791546526]
  Predictive coding (PC) has shown promising performance in machine intelligence tasks. PC can model information processing in different brain areas, can be used in cognitive control and robotics.
  arXiv  Detail & Related papers  (2023-08-15T16:37:16Z)
• Sources of Uncertainty in Machine Learning -- A Statisticians' View [3.1498833540989413]
  The paper aims to formalize the two types of uncertainty associated with machine learning. Drawing parallels between statistical concepts and uncertainty in machine learning, we also demonstrate the role of data and their influence on uncertainty.
  arXiv  Detail & Related papers  (2023-05-26T07:44:19Z)
• Uncertainty Estimation by Fisher Information-based Evidential Deep Learning [61.94125052118442]
  Uncertainty estimation is a key factor that makes deep learning reliable in practical applications. We propose a novel method, Fisher Information-based Evidential Deep Learning ($mathcalI$-EDL) In particular, we introduce Fisher Information Matrix (FIM) to measure the informativeness of evidence carried by each sample, according to which we can dynamically reweight the objective loss terms to make the network more focused on the representation learning of uncertain classes.
  arXiv  Detail & Related papers  (2023-03-03T16:12:59Z)
• Generalized Uncertainty of Deep Neural Networks: Taxonomy and Applications [1.9671123873378717]
  We show that the uncertainty of deep neural networks is not only important in a sense of interpretability and transparency, but also crucial in further advancing their performance. We will generalize the definition of the uncertainty of deep neural networks to any number or vector that is associated with an input or an input-label pair, and catalog existing methods on mining'' such uncertainty from a deep model.
  arXiv  Detail & Related papers  (2023-02-02T22:02:33Z)
• From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligence [113.06484656032978]
  Article argues that embodied intelligence is a key driver for the advancement of machine learning technology. We highlight challenges and opportunities specific to embodied intelligence. We propose research directions which may significantly advance the state-of-the-art in robot learning.
  arXiv  Detail & Related papers  (2021-10-28T16:04:01Z)
• Multi Agent System for Machine Learning Under Uncertainty in Cyber Physical Manufacturing System [78.60415450507706]
  Recent advancements in predictive machine learning has led to its application in various use cases in manufacturing. Most research focused on maximising predictive accuracy without addressing the uncertainty associated with it. In this paper, we determine the sources of uncertainty in machine learning and establish the success criteria of a machine learning system to function well under uncertainty.
  arXiv  Detail & Related papers  (2021-07-28T10:28:05Z)
• Dos and Don'ts of Machine Learning in Computer Security [74.1816306998445]
  Despite great potential, machine learning in security is prone to subtle pitfalls that undermine its performance. We identify common pitfalls in the design, implementation, and evaluation of learning-based security systems. We propose actionable recommendations to support researchers in avoiding or mitigating the pitfalls where possible.
  arXiv  Detail & Related papers  (2020-10-19T13:09:31Z)
• Aleatoric and Epistemic Uncertainty with Random Forests [3.1410342959104725]
  We show how two approaches for measuring the learner's aleatoric and epistemic uncertainty in a prediction can be instantiated with decision trees and random forests. In this paper, we also compare random forests with deep neural networks, which have been used for a similar purpose.
  arXiv  Detail & Related papers  (2020-01-03T17:08:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.