Random-Set Convolutional Neural Network (RS-CNN) for Epistemic Deep
Learning
- URL: http://arxiv.org/abs/2307.05772v1
- Date: Tue, 11 Jul 2023 20:00:35 GMT
- Title: Random-Set Convolutional Neural Network (RS-CNN) for Epistemic Deep
Learning
- Authors: Shireen Kudukkil Manchingal, Muhammad Mubashar, Kaizheng Wang, Keivan
Shariatmadar, Fabio Cuzzolin
- Abstract summary: We propose a novel Random-Set Convolutional Neural Network (RS-CNN) for classification which predicts belief functions rather than probability vectors over the set of classes.
We experimentally demonstrate how our approach outperforms competing uncertainty-aware approaches in a classical evaluation setting.
- Score: 6.1614313137673875
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Machine learning is increasingly deployed in safety-critical domains where
robustness against adversarial attacks is crucial and erroneous predictions
could lead to potentially catastrophic consequences. This highlights the need
for learning systems to be equipped with the means to determine a model's
confidence in its prediction and the epistemic uncertainty associated with it,
'to know when a model does not know'. In this paper, we propose a novel
Random-Set Convolutional Neural Network (RS-CNN) for classification which
predicts belief functions rather than probability vectors over the set of
classes, using the mathematics of random sets, i.e., distributions over the
power set of the sample space. Based on the epistemic deep learning approach,
random-set models are capable of representing the 'epistemic' uncertainty
induced in machine learning by limited training sets. We estimate epistemic
uncertainty by approximating the size of credal sets associated with the
predicted belief functions, and experimentally demonstrate how our approach
outperforms competing uncertainty-aware approaches in a classical evaluation
setting. The performance of RS-CNN is best demonstrated on OOD samples where it
manages to capture the true prediction while standard CNNs fail.
Related papers
- Detecting Model Misspecification in Amortized Bayesian Inference with Neural Networks: An Extended Investigation [9.950524371154394]
We propose a new misspecification measure that can be trained in an unsupervised fashion and reliably detects model misspecification at test time.
We show how the proposed misspecification test warns users about suspicious outputs, raises an alarm when predictions are not trustworthy, and guides model designers in their search for better simulators.
arXiv Detail & Related papers (2024-06-05T11:30:16Z) - Evidential Deep Learning: Enhancing Predictive Uncertainty Estimation
for Earth System Science Applications [0.32302664881848275]
Evidential deep learning is a technique that extends parametric deep learning to higher-order distributions.
This study compares the uncertainty derived from evidential neural networks to those obtained from ensembles.
We show evidential deep learning models attaining predictive accuracy rivaling standard methods, while robustly quantifying both sources of uncertainty.
arXiv Detail & Related papers (2023-09-22T23:04:51Z) - Toward Robust Uncertainty Estimation with Random Activation Functions [3.0586855806896045]
We propose a novel approach for uncertainty quantification via ensembles, called Random Activation Functions (RAFs) Ensemble.
RAFs Ensemble outperforms state-of-the-art ensemble uncertainty quantification methods on both synthetic and real-world datasets.
arXiv Detail & Related papers (2023-02-28T13:17:56Z) - Interpretable Self-Aware Neural Networks for Robust Trajectory
Prediction [50.79827516897913]
We introduce an interpretable paradigm for trajectory prediction that distributes the uncertainty among semantic concepts.
We validate our approach on real-world autonomous driving data, demonstrating superior performance over state-of-the-art baselines.
arXiv Detail & Related papers (2022-11-16T06:28:20Z) - Epistemic Deep Learning [6.144873990390373]
We introduce a concept called epistemic deep learning based on the random-set interpretation of belief functions.
We propose a novel random-set convolutional neural network for classification that produces scores for sets of classes by learning set-valued ground truth representations.
arXiv Detail & Related papers (2022-06-15T15:39:52Z) - The Unreasonable Effectiveness of Deep Evidential Regression [72.30888739450343]
A new approach with uncertainty-aware regression-based neural networks (NNs) shows promise over traditional deterministic methods and typical Bayesian NNs.
We detail the theoretical shortcomings and analyze the performance on synthetic and real-world data sets, showing that Deep Evidential Regression is a quantification rather than an exact uncertainty.
arXiv Detail & Related papers (2022-05-20T10:10:32Z) - NUQ: Nonparametric Uncertainty Quantification for Deterministic Neural
Networks [151.03112356092575]
We show the principled way to measure the uncertainty of predictions for a classifier based on Nadaraya-Watson's nonparametric estimate of the conditional label distribution.
We demonstrate the strong performance of the method in uncertainty estimation tasks on a variety of real-world image datasets.
arXiv Detail & Related papers (2022-02-07T12:30:45Z) - Dense Uncertainty Estimation [62.23555922631451]
In this paper, we investigate neural networks and uncertainty estimation techniques to achieve both accurate deterministic prediction and reliable uncertainty estimation.
We work on two types of uncertainty estimations solutions, namely ensemble based methods and generative model based methods, and explain their pros and cons while using them in fully/semi/weakly-supervised framework.
arXiv Detail & Related papers (2021-10-13T01:23:48Z) - CC-Cert: A Probabilistic Approach to Certify General Robustness of
Neural Networks [58.29502185344086]
In safety-critical machine learning applications, it is crucial to defend models against adversarial attacks.
It is important to provide provable guarantees for deep learning models against semantically meaningful input transformations.
We propose a new universal probabilistic certification approach based on Chernoff-Cramer bounds.
arXiv Detail & Related papers (2021-09-22T12:46:04Z) - Failure Prediction by Confidence Estimation of Uncertainty-Aware
Dirichlet Networks [6.700873164609009]
It is shown that uncertainty-aware deep Dirichlet neural networks provide an improved separation between the confidence of correct and incorrect predictions in the true class probability metric.
A new criterion is proposed for learning the true class probability by matching prediction confidence scores while taking imbalance and TCP constraints into account.
arXiv Detail & Related papers (2020-10-19T21:06:45Z) - Frequentist Uncertainty in Recurrent Neural Networks via Blockwise
Influence Functions [121.10450359856242]
Recurrent neural networks (RNNs) are instrumental in modelling sequential and time-series data.
Existing approaches for uncertainty quantification in RNNs are based predominantly on Bayesian methods.
We develop a frequentist alternative that: (a) does not interfere with model training or compromise its accuracy, (b) applies to any RNN architecture, and (c) provides theoretical coverage guarantees on the estimated uncertainty intervals.
arXiv Detail & Related papers (2020-06-20T22:45:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.