Abstract: Conventional Bayesian Neural Networks (BNNs) are known to be capable of
providing multiple outputs for a single input, the variations in which can be
utilised to detect Out of Distribution (OOD) inputs. BNNs are difficult to
train due to their sensitivity towards the choice of priors. To alleviate this
issue, we propose pseudo-BNNs where instead of learning distributions over
weights, we use point estimates and perturb weights at the time of inference.
We modify the cost function of conventional BNNs and use it to learn parameters
for the purpose of injecting right amount of random perturbations to each of
the weights of a neural network with point estimate. In order to effectively
segregate OOD inputs from In Distribution (ID) inputs using multiple outputs,
we further propose two measures, derived from the index of dispersion and
entropy of probability distributions, and combine them with the proposed
pseudo-BNNs. Overall, this combination results in a principled technique to
detect OOD samples at the time of inference. We evaluate our technique on a
wide variety of neural network architectures and image classification datasets.
We observe that our method achieves state of the art results and beats the
related previous work on various metrics such as FPR at 95% TPR, AUROC, AUPR
and Detection Error by just using 2 to 5 samples of weights per input.