Weight Pruning and Uncertainty in Radio Galaxy Classification
- URL: http://arxiv.org/abs/2111.11654v1
- Date: Tue, 23 Nov 2021 05:01:27 GMT
- Title: Weight Pruning and Uncertainty in Radio Galaxy Classification
- Authors: Devina Mohan, Anna Scaife
- Abstract summary: We use variational inference to quantify the degree of uncertainty in model predictions of radio galaxy classification.
We show that the level of model posterior variance for individual test samples is correlated with human uncertainty when labelling radio galaxies.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: In this work we use variational inference to quantify the degree of epistemic
uncertainty in model predictions of radio galaxy classification and show that
the level of model posterior variance for individual test samples is correlated
with human uncertainty when labelling radio galaxies. We explore the model
performance and uncertainty calibration for a variety of different weight
priors and suggest that a sparse prior produces more well-calibrated
uncertainty estimates. Using the posterior distributions for individual
weights, we show that signal-to-noise ratio (SNR) ranking allows pruning of the
fully-connected layers to the level of 30\% without significant loss of
performance, and that this pruning increases the predictive uncertainty in the
model. Finally we show that, like other work in this field, we experience a
cold posterior effect. We examine whether adapting the cost function in our
model to accommodate model misspecification can compensate for this effect, but
find that it does not make a significant difference. We also examine the effect
of principled data augmentation and find that it improves upon the baseline but
does not compensate for the observed effect fully. We interpret this as the
cold posterior effect being due to the overly effective curation of our
training sample leading to likelihood misspecification, and raise this as a
potential issue for Bayesian deep learning approaches to radio galaxy
classification in future.
Related papers
- Identifiable causal inference with noisy treatment and no side information [6.432072145009342]
This study proposes a model that assumes a continuous treatment variable that is inaccurately measured.
We prove that our model's causal effect estimates are identifiable, even without side information and knowledge of the measurement error variance.
Our work extends the range of applications in which reliable causal inference can be conducted.
arXiv Detail & Related papers (2023-06-18T18:38:10Z) - Propagating Variational Model Uncertainty for Bioacoustic Call Label
Smoothing [15.929064190849665]
We focus on using the predictive uncertainty signal calculated by Bayesian neural networks to guide learning in the self-same task the model is being trained on.
Not opting for costly Monte Carlo sampling of weights, we propagate the approximate hidden variance in an end-to-end manner.
We show that, through the explicit usage of the uncertainty in the loss calculation, the variational model is led to improved predictive and calibration performance.
arXiv Detail & Related papers (2022-10-19T13:04:26Z) - Monotonicity and Double Descent in Uncertainty Estimation with Gaussian
Processes [52.92110730286403]
It is commonly believed that the marginal likelihood should be reminiscent of cross-validation metrics and that both should deteriorate with larger input dimensions.
We prove that by tuning hyper parameters, the performance, as measured by the marginal likelihood, improves monotonically with the input dimension.
We also prove that cross-validation metrics exhibit qualitatively different behavior that is characteristic of double descent.
arXiv Detail & Related papers (2022-10-14T08:09:33Z) - On Calibrated Model Uncertainty in Deep Learning [0.0]
We extend the approximate inference for the loss-calibrated Bayesian framework to dropweights based Bayesian neural networks.
We show that decisions informed by loss-calibrated uncertainty can improve diagnostic performance to a greater extent than straightforward alternatives.
arXiv Detail & Related papers (2022-06-15T20:16:32Z) - How Tempering Fixes Data Augmentation in Bayesian Neural Networks [22.188535244056016]
We show that tempering implicitly reduces the misspecification arising from modeling augmentations as i.i.d. data.
The temperature mimics the role of the effective sample size, reflecting the gain in information provided by the augmentations.
arXiv Detail & Related papers (2022-05-27T11:06:56Z) - Quantifying Uncertainty in Deep Learning Approaches to Radio Galaxy
Classification [0.0]
We show that the level of model posterior variance for individual test samples is correlated with human uncertainty when labelling radio galaxies.
We explore the model performance and uncertainty calibration for a variety of different weight priors and suggest that a sparse prior produces more well-calibrated uncertainty estimates.
arXiv Detail & Related papers (2022-01-04T15:31:18Z) - Dense Uncertainty Estimation [62.23555922631451]
In this paper, we investigate neural networks and uncertainty estimation techniques to achieve both accurate deterministic prediction and reliable uncertainty estimation.
We work on two types of uncertainty estimations solutions, namely ensemble based methods and generative model based methods, and explain their pros and cons while using them in fully/semi/weakly-supervised framework.
arXiv Detail & Related papers (2021-10-13T01:23:48Z) - Increasing the efficiency of randomized trial estimates via linear
adjustment for a prognostic score [59.75318183140857]
Estimating causal effects from randomized experiments is central to clinical research.
Most methods for historical borrowing achieve reductions in variance by sacrificing strict type-I error rate control.
arXiv Detail & Related papers (2020-12-17T21:10:10Z) - The Hidden Uncertainty in a Neural Networks Activations [105.4223982696279]
The distribution of a neural network's latent representations has been successfully used to detect out-of-distribution (OOD) data.
This work investigates whether this distribution correlates with a model's epistemic uncertainty, thus indicating its ability to generalise to novel inputs.
arXiv Detail & Related papers (2020-12-05T17:30:35Z) - Unlabelled Data Improves Bayesian Uncertainty Calibration under
Covariate Shift [100.52588638477862]
We develop an approximate Bayesian inference scheme based on posterior regularisation.
We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
arXiv Detail & Related papers (2020-06-26T13:50:19Z) - Learning to Predict Error for MRI Reconstruction [67.76632988696943]
We demonstrate that predictive uncertainty estimated by the current methods does not highly correlate with prediction error.
We propose a novel method that estimates the target labels and magnitude of the prediction error in two steps.
arXiv Detail & Related papers (2020-02-13T15:55:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.