Internal noise in deep neural networks: interplay of depth, neuron number, and noise injection step

• URL: http://arxiv.org/abs/2604.08117v1
• Date: Thu, 09 Apr 2026 11:36:14 GMT
• Title: Internal noise in deep neural networks: interplay of depth, neuron number, and noise injection step
• Abstract summary: This paper examines the influence of internal Gaussian noise on the performance of deep feedforward neural networks.<n>Two scenarios are analyzed: noise introduced before and after the activation function, for both additive and multiplicative noise influence.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper examines the influence of internal Gaussian noise on the performance of deep feedforward neural networks, focusing on the role of the noise injection stage relative to the activation function. Two scenarios are analyzed: noise introduced before and after the activation function, for both additive and multiplicative noise influence. The case of noise before activation function is similar to perturbations in the input channel of neuron, while the noise introduced after activation function is analogous to noise occurring either within the neuron itself or in its output channel. The types of noise and the method of their introduction were inspired by analog neural networks. The results show that the activation function acts as an effective nonlinear filter of noise. Networks with noise introduced before the activation function consistently achieve higher accuracy than those with noise applied after it, with additive noise being more effectively suppressed in this case. For noise introduced after the activation function, multiplicative noise is less detrimental than additive noise, and earlier hidden layers contribute more significantly to performance degradation due to cumulative noise amplification governed by the statistical properties of subsequent weight matrices. The study also demonstrates that pooling-based noise reduction is effective in both cases when noise is introduced before and after the activation function, consistently improving network performance.

Related papers

• Spatial Partial Functionalization of Neural Networks based on Noise Fields [0.0]
  Noise in neural computation is typically regarded as a disturbance, but its spatial distribution may also actively regulate which parts of a network participate in computation.<n>This paper investigates the spatial partial functionalization of Noise-modulated Neural Networks using noise fields.
  arXiv  Detail & Related papers  (2026-06-23T13:50:50Z)
• General aspects of internal noise in spiking neural networks [0.0]
  This study examines the impact of additive and multiplicative noise on a single leaky integrate-and-fire neuron and a trained spiking neural network (SNN)<n>Results show that multiplicative noise applied to the membrane potential has the most detrimental effect on network performance, leading to a significant degradation in accuracy.
  arXiv  Detail & Related papers  (2026-04-15T08:24:42Z)
• Paradoxical noise preference in RNNs [0.0]
  In recurrent neural networks (RNNs) used to model biological neural networks, noise is typically introduced during training to emulate biological variability and regularize learning.<n>We find that continuous-time recurrent neural networks (CTRNNs) often perform best at a nonzero noise level, specifically, the same level used during training.<n>This noise preference typically arises when noise is injected inside the neural activation function; networks trained with noise injected outside the activation function perform best with zero noise.
  arXiv  Detail & Related papers  (2026-01-08T03:11:51Z)
• Mixture of Noise for Pre-Trained Model-Based Class-Incremental Learning [59.635264288605946]
  Class Incremental Learning (CIL) aims to continuously learn new categories while retaining the knowledge of old ones.<n>Existing approaches that apply lightweight fine-tuning to backbones still induce drift.<n>We propose Mixture of Noise (Min) to mitigate the degradation of backbone generalization from adapting new tasks.
  arXiv  Detail & Related papers  (2025-09-20T16:07:20Z)
• Impact of internal noise on convolutional neural networks [0.0]
  We study the impact of noise on a simplified trained convolutional network.<n>The propagation of uncorrelated noise depends on the statistical properties of the connection matrix.<n>An analysis of the noise level in the network's output signal shows a strong correlation with the results of numerical simulations.
  arXiv  Detail & Related papers  (2025-05-10T11:49:37Z)
• Internal noise in hardware deep and recurrent neural networks helps with learning [0.0]
  Internal noise during the training of neural networks affects the final performance of recurrent and deep neural networks.<n>In most cases, both deep and echo state networks benefit from internal noise during training, as it enhances their resilience to noise.
  arXiv  Detail & Related papers  (2025-04-18T16:26:46Z)
• Enhance Vision-Language Alignment with Noise [59.2608298578913]
  We investigate whether the frozen model can be fine-tuned by customized noise.<n>We propose Positive-incentive Noise (PiNI) which can fine-tune CLIP via injecting noise into both visual and text encoders.
  arXiv  Detail & Related papers  (2024-12-14T12:58:15Z)
• Impact of white noise in artificial neural networks trained for classification: performance and noise mitigation strategies [0.0]
  We consider how additive and multiplicative Gaussian white noise on the neuronal level can affect the accuracy of the network. We adapt several noise reduction techniques to the essential setting of classification tasks.
  arXiv  Detail & Related papers  (2024-11-07T01:21:12Z)
• Noise impact on recurrent neural network with linear activation function [0.0]
  We study the peculiarities of internal noise propagation in recurrent ANN on the example of echo state network (ESN) Here we consider the case when artificial neurons have linear activation function with different slope coefficients. We have found that the general view of variance and signal-to-noise ratio of ESN output signal is similar to only one neuron.
  arXiv  Detail & Related papers  (2023-03-23T13:43:05Z)
• Learning Noise via Dynamical Decoupling of Entangled Qubits [49.38020717064383]
  Noise in entangled quantum systems is difficult to characterize due to many-body effects involving multiple degrees of freedom. We develop and apply multi-qubit dynamical decoupling sequences that characterize noise that occurs during two-qubit gates.
  arXiv  Detail & Related papers  (2022-01-26T20:22:38Z)
• Removing Noise from Extracellular Neural Recordings Using Fully Convolutional Denoising Autoencoders [62.997667081978825]
  We propose a Fully Convolutional Denoising Autoencoder, which learns to produce a clean neuronal activity signal from a noisy multichannel input. The experimental results on simulated data show that our proposed method can improve significantly the quality of noise-corrupted neural signals.
  arXiv  Detail & Related papers  (2021-09-18T14:51:24Z)
• Removing Adversarial Noise in Class Activation Feature Space [160.78488162713498]
  We propose to remove adversarial noise by implementing a self-supervised adversarial training mechanism in a class activation feature space. We train a denoising model to minimize the distances between the adversarial examples and the natural examples in the class activation feature space. Empirical evaluations demonstrate that our method could significantly enhance adversarial robustness in comparison to previous state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-04-19T10:42:24Z)
• On Dynamic Noise Influence in Differentially Private Learning [102.6791870228147]
  Private Gradient Descent (PGD) is a commonly used private learning framework, which noises based on the Differential protocol. Recent studies show that emphdynamic privacy schedules can improve at the final iteration, yet yet theoreticals of the effectiveness of such schedules remain limited. This paper provides comprehensive analysis of noise influence in dynamic privacy schedules to answer these critical questions.
  arXiv  Detail & Related papers  (2021-01-19T02:04:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.