Neural Variability Enhances Artificial Network Robustness

• URL: http://arxiv.org/abs/2606.13801v1
• Date: Thu, 11 Jun 2026 18:15:39 GMT
• Title: Neural Variability Enhances Artificial Network Robustness
• Abstract summary: Noise and signal correlations may be optimized for discrimination in animals, whereas artificial neural network (ANN) studies have shown similar benefits of noise in machine learning tasks.<n>Here we investigate whether correlated noise improves the robustness of artificial neural networks to adversarial attacks and naturalistic image modifications.
• Score: 3.314766137078935
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural responses in cortex exhibit substantial trial-to-trial variability in response to repeated stimuli, while peripheral sensory neurons respond far more consistently, leading many to wonder whether stochasticity may carry meaning. Existing work has argued that noise and signal correlations may be optimized for discrimination in animals, whereas artificial neural network (ANN) studies have shown similar benefits of noise in machine learning tasks, although most ANN work has neglected the effects of correlations. Here we investigate whether correlated noise improves the robustness of artificial neural networks to adversarial attacks and naturalistic image modifications. Using the covariance of activations under modified versus clean inputs, we find that structured noise may significantly improve network robustness. Robustness to naturalistic image modifications benefits most from structure, but this structure transfers poorly across modification types. In contrast, noise structure from adversarial attacks can generalize to other kinds of attacks. These results suggest that structured noise in ANN activations generally improves robustness, establishing a biologically plausible strategy for creating robust artificial neural networks that only relies on local information.

Related papers

• General Self-Prediction Enhancement for Spiking Neurons [71.01912385372577]
  Spiking Neural Networks (SNNs) are highly energy-efficient due to event-driven, sparse computation, but their training is challenged by spike non-differentiability and trade-offs among performance, efficiency, and biological plausibility.<n>We propose a self-prediction enhanced spiking neuron method that generates an internal prediction current from its input-output history to modulate membrane potential.<n>This design offers dual advantages, it creates a continuous gradient path that alleviates vanishing gradients and boosts training stability and accuracy, while also aligning with biological principles, which resembles distal dendritic modulation and error-driven synaptic plasticity.
  arXiv  Detail & Related papers  (2026-01-29T15:08:48Z)
• Paradoxical noise preference in RNNs [0.0]
  In recurrent neural networks (RNNs) used to model biological neural networks, noise is typically introduced during training to emulate biological variability and regularize learning.<n>We find that continuous-time recurrent neural networks (CTRNNs) often perform best at a nonzero noise level, specifically, the same level used during training.<n>This noise preference typically arises when noise is injected inside the neural activation function; networks trained with noise injected outside the activation function perform best with zero noise.
  arXiv  Detail & Related papers  (2026-01-08T03:11:51Z)
• Credit Assignment via Neural Manifold Noise Correlation [0.0]
  Credit assignment is central to learning in brains and machines.<n>Noise correlation estimates gradients by correlating perturbations of activity with changes in output.<n>We propose neural manifold noise correlation, which performs credit assignment using perturbations restricted to the neural manifold.
  arXiv  Detail & Related papers  (2026-01-06T01:17:55Z)
• NCCR: to Evaluate the Robustness of Neural Networks and Adversarial Examples [11.038285179317114]
  In this work, we propose a metric called the neuron cover change rate (NCCR) to measure the ability of deep learning models to resist attacks.<n>NCCR monitors alterations in the output of specifically chosen neurons when the input is perturbed.<n>The results of the experiment on image recognition and the speaker recognition model show that our metrics can provide a good assessment of the robustness of neural networks or their inputs.
  arXiv  Detail & Related papers  (2025-07-29T03:58:20Z)
• Internal noise in hardware deep and recurrent neural networks helps with learning [0.0]
  Internal noise during the training of neural networks affects the final performance of recurrent and deep neural networks.<n>In most cases, both deep and echo state networks benefit from internal noise during training, as it enhances their resilience to noise.
  arXiv  Detail & Related papers  (2025-04-18T16:26:46Z)
• A Tunable Despeckling Neural Network Stabilized via Diffusion Equation [15.996302571895045]
  Adrialversa attacks can be used as a criterion for judging the adaptability of neural networks to real data.<n>We propose a tunable, regularized neural network framework that unrolls a shallow denoising neural network block and a diffusion regularity block into a single network for end-to-end training.
  arXiv  Detail & Related papers  (2024-11-24T17:08:43Z)
• Impact of white noise in artificial neural networks trained for classification: performance and noise mitigation strategies [0.0]
  We consider how additive and multiplicative Gaussian white noise on the neuronal level can affect the accuracy of the network. We adapt several noise reduction techniques to the essential setting of classification tasks.
  arXiv  Detail & Related papers  (2024-11-07T01:21:12Z)
• Exploring neural oscillations during speech perception via surrogate gradient spiking neural networks [59.38765771221084]
  We present a physiologically inspired speech recognition architecture compatible and scalable with deep learning frameworks. We show end-to-end gradient descent training leads to the emergence of neural oscillations in the central spiking neural network. Our findings highlight the crucial inhibitory role of feedback mechanisms, such as spike frequency adaptation and recurrent connections, in regulating and synchronising neural activity to improve recognition performance.
  arXiv  Detail & Related papers  (2024-04-22T09:40:07Z)
• Graph Neural Networks for Learning Equivariant Representations of Neural Networks [55.04145324152541]
  We propose to represent neural networks as computational graphs of parameters. Our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations.
  arXiv  Detail & Related papers  (2024-03-18T18:01:01Z)
• Learning Provably Robust Estimators for Inverse Problems via Jittering [51.467236126126366]
  We investigate whether jittering, a simple regularization technique, is effective for learning worst-case robust estimators for inverse problems. We show that jittering significantly enhances the worst-case robustness, but can be suboptimal for inverse problems beyond denoising.
  arXiv  Detail & Related papers  (2023-07-24T14:19:36Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Artificial Neural Variability for Deep Learning: On Overfitting, Noise Memorization, and Catastrophic Forgetting [135.0863818867184]
  artificial neural variability (ANV) helps artificial neural networks learn some advantages from natural'' neural networks. ANV plays as an implicit regularizer of the mutual information between the training data and the learned model. It can effectively relieve overfitting, label noise memorization, and catastrophic forgetting at negligible costs.
  arXiv  Detail & Related papers  (2020-11-12T06:06:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.