Abstract: Deep neural networks are increasingly being used for the analysis of medical
images. However, most works neglect the uncertainty in the model's prediction.
We propose an uncertainty-aware deep kernel learning model which permits the
estimation of the uncertainty in the prediction by a pipeline of a
Convolutional Neural Network and a sparse Gaussian Process. Furthermore, we
adapt different pre-training methods to investigate their impacts on the
proposed model. We apply our approach to Bone Age Prediction and Lesion
Localization. In most cases, the proposed model shows better performance
compared to common architectures. More importantly, our model expresses
systematically higher confidence in more accurate predictions and less
confidence in less accurate ones. Our model can also be used to detect
challenging and controversial test samples. Compared to related methods such as
Monte-Carlo Dropout, our approach derives the uncertainty information in a
purely analytical fashion and is thus computationally more efficient.