ZigZag: Universal Sampling-free Uncertainty Estimation Through Two-Step Inference

• URL: http://arxiv.org/abs/2211.11435v3
• Date: Sun, 26 May 2024 21:10:08 GMT
• Title: ZigZag: Universal Sampling-free Uncertainty Estimation Through Two-Step Inference
• Authors: Nikita Durasov, Nik Dorndorf, Hieu Le, Pascal Fua,
• Abstract summary: We introduce a sampling-free approach that is generic and easy to deploy. We produce reliable uncertainty estimates on par with state-of-the-art methods at a significantly lower computational cost.
• Score: 54.17205151960878
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Whereas the ability of deep networks to produce useful predictions has been amply demonstrated, estimating the reliability of these predictions remains challenging. Sampling approaches such as MC-Dropout and Deep Ensembles have emerged as the most popular ones for this purpose. Unfortunately, they require many forward passes at inference time, which slows them down. Sampling-free approaches can be faster but suffer from other drawbacks, such as lower reliability of uncertainty estimates, difficulty of use, and limited applicability to different types of tasks and data. In this work, we introduce a sampling-free approach that is generic and easy to deploy, while producing reliable uncertainty estimates on par with state-of-the-art methods at a significantly lower computational cost. It is predicated on training the network to produce the same output with and without additional information about it. At inference time, when no prior information is given, we use the network's own prediction as the additional information. We then take the distance between the predictions with and without prior information as our uncertainty measure. We demonstrate our approach on several classification and regression tasks. We show that it delivers results on par with those of Ensembles but at a much lower computational cost.

Related papers

• Estimating Uncertainty with Implicit Quantile Network [0.0]
  Uncertainty quantification is an important part of many performance critical applications. This paper provides a simple alternative to existing approaches such as ensemble learning and bayesian neural networks.
  arXiv  Detail & Related papers  (2024-08-26T13:33:14Z)
• Extracting Usable Predictions from Quantized Networks through Uncertainty Quantification for OOD Detection [0.0]
  OOD detection has become more pertinent with advances in network design and increased task complexity. We introduce an Uncertainty Quantification(UQ) technique to quantify the uncertainty in the predictions from a pre-trained vision model. We observe that our technique saves up to 80% of ignored samples from being misclassified.
  arXiv  Detail & Related papers  (2024-03-02T03:03:29Z)
• One step closer to unbiased aleatoric uncertainty estimation [71.55174353766289]
  We propose a new estimation method by actively de-noising the observed data. By conducting a broad range of experiments, we demonstrate that our proposed approach provides a much closer approximation to the actual data uncertainty than the standard method.
  arXiv  Detail & Related papers  (2023-12-16T14:59:11Z)
• Pedestrian Trajectory Forecasting Using Deep Ensembles Under Sensing Uncertainty [125.41260574344933]
  We consider an encoder-decoder based deep ensemble network for capturing both perception and predictive uncertainty simultaneously. Overall, deep ensembles provided more robust predictions and the consideration of upstream uncertainty further increased the estimation accuracy for the model.
  arXiv  Detail & Related papers  (2023-05-26T04:27:48Z)
• Training Uncertainty-Aware Classifiers with Conformalized Deep Learning [7.837881800517111]
  Deep neural networks are powerful tools to detect hidden patterns in data and leverage them to make predictions, but they are not designed to understand uncertainty. We develop a novel training algorithm that can lead to more dependable uncertainty estimates, without sacrificing predictive power.
  arXiv  Detail & Related papers  (2022-05-12T05:08:10Z)
• Non-Stationary Bandit Learning via Predictive Sampling [15.88678122212934]
  We show that Thompson sampling can perform poorly in non-stationary environments. We propose predictive sampling, an algorithm that deprioritizes acquiring information that quickly loses usefulness.
  arXiv  Detail & Related papers  (2022-05-04T09:37:16Z)
• CertainNet: Sampling-free Uncertainty Estimation for Object Detection [65.28989536741658]
  Estimating the uncertainty of a neural network plays a fundamental role in safety-critical settings. In this work, we propose a novel sampling-free uncertainty estimation method for object detection. We call it CertainNet, and it is the first to provide separate uncertainties for each output signal: objectness, class, location and size.
  arXiv  Detail & Related papers  (2021-10-04T17:59:31Z)
• A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification [1.90365714903665]
  This hands-on introduction is aimed at a reader interested in the practical implementation of distribution-free UQ. We will include many explanatory illustrations, examples, and code samples in Python, with PyTorch syntax.
  arXiv  Detail & Related papers  (2021-07-15T17:59:50Z)
• Quantifying Uncertainty in Deep Spatiotemporal Forecasting [67.77102283276409]
  We describe two types of forecasting problems: regular grid-based and graph-based. We analyze UQ methods from both the Bayesian and the frequentist point view, casting in a unified framework via statistical decision theory. Through extensive experiments on real-world road network traffic, epidemics, and air quality forecasting tasks, we reveal the statistical computational trade-offs for different UQ methods.
  arXiv  Detail & Related papers  (2021-05-25T14:35:46Z)
• Ambiguity in Sequential Data: Predicting Uncertain Futures with Recurrent Models [110.82452096672182]
  We propose an extension of the Multiple Hypothesis Prediction (MHP) model to handle ambiguous predictions with sequential data. We also introduce a novel metric for ambiguous problems, which is better suited to account for uncertainties.
  arXiv  Detail & Related papers  (2020-03-10T09:15:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.