Incorporating Unlabelled Data into Bayesian Neural Networks
- URL: http://arxiv.org/abs/2304.01762v2
- Date: Fri, 19 May 2023 14:23:39 GMT
- Title: Incorporating Unlabelled Data into Bayesian Neural Networks
- Authors: Mrinank Sharma, Tom Rainforth, Yee Whye Teh, Vincent Fortuin
- Abstract summary: We introduce Self-Supervised Bayesian Neural Networks, which use unlabelled data to learn improved prior predictive distributions.
We show that self-supervised prior predictives capture image semantics better than conventional BNN priors.
- Score: 60.51580870352031
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Conventional Bayesian Neural Networks (BNNs) cannot leverage unlabelled data
to improve their predictions. To overcome this limitation, we introduce
Self-Supervised Bayesian Neural Networks, which use unlabelled data to learn
improved prior predictive distributions by maximising an evidence lower bound
during an unsupervised pre-training step. With a novel methodology developed to
better understand prior predictive distributions, we then show that
self-supervised prior predictives capture image semantics better than
conventional BNN priors. In our empirical evaluations, we see that
self-supervised BNNs offer the label efficiency of self-supervised methods and
the uncertainty estimates of Bayesian methods, particularly outperforming
conventional BNNs in low-to-medium data regimes.
Related papers
- Bayesian Neural Networks with Domain Knowledge Priors [52.80929437592308]
We propose a framework for integrating general forms of domain knowledge into a BNN prior.
We show that BNNs using our proposed domain knowledge priors outperform those with standard priors.
arXiv Detail & Related papers (2024-02-20T22:34:53Z) - Leveraging Unlabeled Data for 3D Medical Image Segmentation through
Self-Supervised Contrastive Learning [3.7395287262521717]
Current 3D semi-supervised segmentation methods face significant challenges such as limited consideration of contextual information.
We introduce two distinctworks designed to explore and exploit the discrepancies between them, ultimately correcting the erroneous prediction results.
We employ a self-supervised contrastive learning paradigm to distinguish between reliable and unreliable predictions.
arXiv Detail & Related papers (2023-11-21T14:03:16Z) - Interpretable Self-Aware Neural Networks for Robust Trajectory
Prediction [50.79827516897913]
We introduce an interpretable paradigm for trajectory prediction that distributes the uncertainty among semantic concepts.
We validate our approach on real-world autonomous driving data, demonstrating superior performance over state-of-the-art baselines.
arXiv Detail & Related papers (2022-11-16T06:28:20Z) - Posterior Regularized Bayesian Neural Network Incorporating Soft and
Hard Knowledge Constraints [12.050265348673078]
We propose a novel Posterior-Regularized Bayesian Neural Network (PR-BNN) model by incorporating different types of knowledge constraints.
Experiments in simulation and two case studies about aviation landing prediction and solar energy output prediction have shown the knowledge constraints and the performance improvement of the proposed model.
arXiv Detail & Related papers (2022-10-16T18:58:50Z) - Out of Distribution Data Detection Using Dropout Bayesian Neural
Networks [29.84998820573774]
We first show how previous attempts to leverage the randomized embeddings induced by the intermediate layers of a dropout BNN can fail due to the distance metric used.
We introduce an alternative approach to measuring embedding uncertainty, justify its use theoretically, and demonstrate how incorporating embedding uncertainty improves OOD data identification across three tasks: image classification, language classification, and malware detection.
arXiv Detail & Related papers (2022-02-18T02:23:43Z) - S2-BNN: Bridging the Gap Between Self-Supervised Real and 1-bit Neural
Networks via Guided Distribution Calibration [74.5509794733707]
We present a novel guided learning paradigm from real-valued to distill binary networks on the final prediction distribution.
Our proposed method can boost the simple contrastive learning baseline by an absolute gain of 5.515% on BNNs.
Our method achieves substantial improvement over the simple contrastive learning baseline, and is even comparable to many mainstream supervised BNN methods.
arXiv Detail & Related papers (2021-02-17T18:59:28Z) - Unlabelled Data Improves Bayesian Uncertainty Calibration under
Covariate Shift [100.52588638477862]
We develop an approximate Bayesian inference scheme based on posterior regularisation.
We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
arXiv Detail & Related papers (2020-06-26T13:50:19Z) - Frequentist Uncertainty in Recurrent Neural Networks via Blockwise
Influence Functions [121.10450359856242]
Recurrent neural networks (RNNs) are instrumental in modelling sequential and time-series data.
Existing approaches for uncertainty quantification in RNNs are based predominantly on Bayesian methods.
We develop a frequentist alternative that: (a) does not interfere with model training or compromise its accuracy, (b) applies to any RNN architecture, and (c) provides theoretical coverage guarantees on the estimated uncertainty intervals.
arXiv Detail & Related papers (2020-06-20T22:45:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.