Uncertainty Quantification With Noise Injection in Neural Networks: A Bayesian Perspective

• URL: http://arxiv.org/abs/2501.12314v1
• Date: Tue, 21 Jan 2025 17:28:52 GMT
• Title: Uncertainty Quantification With Noise Injection in Neural Networks: A Bayesian Perspective
• Authors: Xueqiong Yuan, Jipeng Li, Ercan Engin Kuruoglu,
• Abstract summary: We establish a connection between noise injection and uncertainty quantification from a Bayesian standpoint. We introduce a Monte Carlo Noise Injection (MCNI) method, which involves injecting noise into the parameters during training. Our method demonstrates superior performance compared to the baseline model.
• Score: 1.6044444452278062
• License:
• Abstract: Model uncertainty quantification involves measuring and evaluating the uncertainty linked to a model's predictions, helping assess their reliability and confidence. Noise injection is a technique used to enhance the robustness of neural networks by introducing randomness. In this paper, we establish a connection between noise injection and uncertainty quantification from a Bayesian standpoint. We theoretically demonstrate that injecting noise into the weights of a neural network is equivalent to Bayesian inference on a deep Gaussian process. Consequently, we introduce a Monte Carlo Noise Injection (MCNI) method, which involves injecting noise into the parameters during training and performing multiple forward propagations during inference to estimate the uncertainty of the prediction. Through simulation and experiments on regression and classification tasks, our method demonstrates superior performance compared to the baseline model.

Related papers

• Tractable Function-Space Variational Inference in Bayesian Neural Networks [72.97620734290139]
  A popular approach for estimating the predictive uncertainty of neural networks is to define a prior distribution over the network parameters. We propose a scalable function-space variational inference method that allows incorporating prior information. We show that the proposed method leads to state-of-the-art uncertainty estimation and predictive performance on a range of prediction tasks.
  arXiv  Detail & Related papers  (2023-12-28T18:33:26Z)
• Integrating Uncertainty into Neural Network-based Speech Enhancement [27.868722093985006]
  Supervised masking approaches in the time-frequency domain aim to employ deep neural networks to estimate a multiplicative mask to extract clean speech. This leads to a single estimate for each input without any guarantees or measures of reliability. We study the benefits of modeling uncertainty in clean speech estimation.
  arXiv  Detail & Related papers  (2023-05-15T15:55:12Z)
• Neural State-Space Models: Empirical Evaluation of Uncertainty Quantification [0.0]
  This paper presents preliminary results on uncertainty quantification for system identification with neural state-space models. We frame the learning problem in a Bayesian probabilistic setting and obtain posterior distributions for the neural network's weights and outputs. Based on the posterior, we construct credible intervals on the outputs and define a surprise index which can effectively diagnose usage of the model in a potentially dangerous out-of-distribution regime.
  arXiv  Detail & Related papers  (2023-04-13T08:57:33Z)
• Uncertainty Estimation in Deep Speech Enhancement Using Complex Gaussian Mixture Models [19.442685015494316]
  Single-channel deep speech enhancement approaches often estimate a single multiplicative mask to extract clean speech without a measure of its accuracy. We propose to quantify the uncertainty associated with clean speech estimates in neural network-based speech enhancement.
  arXiv  Detail & Related papers  (2022-12-09T13:03:09Z)
• Variational Neural Networks [88.24021148516319]
  We propose a method for uncertainty estimation in neural networks called Variational Neural Network (VNN) VNN generates parameters for the output distribution of a layer by transforming its inputs with learnable sub-layers. In uncertainty quality estimation experiments, we show that VNNs achieve better uncertainty quality than Monte Carlo Dropout or Bayes By Backpropagation methods.
  arXiv  Detail & Related papers  (2022-07-04T15:41:02Z)
• Density Regression and Uncertainty Quantification with Bayesian Deep Noise Neural Networks [4.376565880192482]
  Deep neural network (DNN) models have achieved state-of-the-art predictive accuracy in a wide range of supervised learning applications. accurately quantifying the uncertainty in DNN predictions remains a challenging task. We propose the Bayesian Deep Noise Neural Network (B-DeepNoise), which generalizes standard Bayesian DNNs by extending the random noise variable to all hidden layers. We evaluate B-DeepNoise against existing methods on benchmark regression datasets, demonstrating its superior performance in terms of prediction accuracy, uncertainty quantification accuracy, and uncertainty quantification efficiency.
  arXiv  Detail & Related papers  (2022-06-12T02:47:29Z)
• NUQ: Nonparametric Uncertainty Quantification for Deterministic Neural Networks [151.03112356092575]
  We show the principled way to measure the uncertainty of predictions for a classifier based on Nadaraya-Watson's nonparametric estimate of the conditional label distribution. We demonstrate the strong performance of the method in uncertainty estimation tasks on a variety of real-world image datasets.
  arXiv  Detail & Related papers  (2022-02-07T12:30:45Z)
• Dense Uncertainty Estimation [62.23555922631451]
  In this paper, we investigate neural networks and uncertainty estimation techniques to achieve both accurate deterministic prediction and reliable uncertainty estimation. We work on two types of uncertainty estimations solutions, namely ensemble based methods and generative model based methods, and explain their pros and cons while using them in fully/semi/weakly-supervised framework.
  arXiv  Detail & Related papers  (2021-10-13T01:23:48Z)
• Sampling-free Variational Inference for Neural Networks with Multiplicative Activation Noise [51.080620762639434]
  We propose a more efficient parameterization of the posterior approximation for sampling-free variational inference. Our approach yields competitive results for standard regression problems and scales well to large-scale image classification tasks.
  arXiv  Detail & Related papers  (2021-03-15T16:16:18Z)
• Leveraging Global Parameters for Flow-based Neural Posterior Estimation [90.21090932619695]
  Inferring the parameters of a model based on experimental observations is central to the scientific method. A particularly challenging setting is when the model is strongly indeterminate, i.e., when distinct sets of parameters yield identical observations. We present a method for cracking such indeterminacy by exploiting additional information conveyed by an auxiliary set of observations sharing global parameters.
  arXiv  Detail & Related papers  (2021-02-12T12:23:13Z)
• A deep-learning based Bayesian approach to seismic imaging and uncertainty quantification [0.4588028371034407]
  Uncertainty is essential when dealing with ill-conditioned inverse problems. It is often not possible to formulate a prior distribution that precisely encodes our prior knowledge about the unknown. We propose to use the functional form of a randomly convolutional neural network as an implicit structured prior.
  arXiv  Detail & Related papers  (2020-01-13T23:46:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.