Noise mitigation strategies in physical feedforward neural networks

• URL: http://arxiv.org/abs/2204.09461v1
• Date: Wed, 20 Apr 2022 13:51:46 GMT
• Title: Noise mitigation strategies in physical feedforward neural networks
• Authors: Nadezhda Semenova and Daniel Brunner
• Abstract summary: Physical neural networks are promising candidates for next generation artificial intelligence hardware. We introduce connectivity topologies, ghost neurons as well as pooling as noise mitigation strategies. We demonstrate the effectiveness of the combined methods based on a fully trained neural network classifying the MNIST handwritten digits.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Physical neural networks are promising candidates for next generation artificial intelligence hardware. In such architectures, neurons and connections are physically realized and do not leverage digital, i.e. practically infinite signal-to-noise ratio digital concepts. They therefore are prone to noise, and base don analytical derivations we here introduce connectivity topologies, ghost neurons as well as pooling as noise mitigation strategies. Finally, we demonstrate the effectiveness of the combined methods based on a fully trained neural network classifying the MNIST handwritten digits.

Related papers

• Impact of white noise in artificial neural networks trained for classification: performance and noise mitigation strategies [0.0]
  We consider how additive and multiplicative Gaussian white noise on the neuronal level can affect the accuracy of the network. We adapt several noise reduction techniques to the essential setting of classification tasks.
  arXiv  Detail & Related papers  (2024-11-07T01:21:12Z)
• Contrastive Learning in Memristor-based Neuromorphic Systems [55.11642177631929]
  Spiking neural networks have become an important family of neuron-based models that sidestep many of the key limitations facing modern-day backpropagation-trained deep networks. In this work, we design and investigate a proof-of-concept instantiation of contrastive-signal-dependent plasticity (CSDP), a neuromorphic form of forward-forward-based, backpropagation-free learning.
  arXiv  Detail & Related papers  (2024-09-17T04:48:45Z)
• Graph Neural Networks for Learning Equivariant Representations of Neural Networks [55.04145324152541]
  We propose to represent neural networks as computational graphs of parameters. Our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations.
  arXiv  Detail & Related papers  (2024-03-18T18:01:01Z)
• Spiking neural network for nonlinear regression [68.8204255655161]
  Spiking neural networks carry the potential for a massive reduction in memory and energy consumption. They introduce temporal and neuronal sparsity, which can be exploited by next-generation neuromorphic hardware. A framework for regression using spiking neural networks is proposed.
  arXiv  Detail & Related papers  (2022-10-06T13:04:45Z)
• Searching for the Essence of Adversarial Perturbations [73.96215665913797]
  We show that adversarial perturbations contain human-recognizable information, which is the key conspirator responsible for a neural network's erroneous prediction. This concept of human-recognizable information allows us to explain key features related to adversarial perturbations.
  arXiv  Detail & Related papers  (2022-05-30T18:04:57Z)
• Stochastic resonance neurons in artificial neural networks [0.0]
  We propose a new type of neural networks using resonances as an inherent part of the architecture. We show that such a neural network is more robust against the impact of noise.
  arXiv  Detail & Related papers  (2022-05-06T18:42:36Z)
• POPPINS : A Population-Based Digital Spiking Neuromorphic Processor with Integer Quadratic Integrate-and-Fire Neurons [50.591267188664666]
  We propose a population-based digital spiking neuromorphic processor in 180nm process technology with two hierarchy populations. The proposed approach enables the developments of biomimetic neuromorphic system and various low-power, and low-latency inference processing applications.
  arXiv  Detail & Related papers  (2022-01-19T09:26:34Z)
• Physical Constraint Embedded Neural Networks for inference and noise regulation [0.0]
  We present methods for embedding even--odd symmetries and conservation laws in neural networks. We demonstrate that it can accurately infer symmetries without prior knowledge. We highlight the noise resilient properties of physical constraint embedded neural networks.
  arXiv  Detail & Related papers  (2021-05-19T14:07:20Z)
• Understanding and mitigating noise in trained deep neural networks [0.0]
  We study the propagation of noise in deep neural networks comprising noisy nonlinear neurons in trained fully connected layers. We find that noise accumulation is generally bound, and adding additional network layers does not worsen the signal to noise ratio beyond a limit. We identify criteria allowing engineers to design noise-resilient novel neural network hardware.
  arXiv  Detail & Related papers  (2021-03-12T17:16:26Z)
• ResiliNet: Failure-Resilient Inference in Distributed Neural Networks [56.255913459850674]
  We introduce ResiliNet, a scheme for making inference in distributed neural networks resilient to physical node failures. Failout simulates physical node failure conditions during training using dropout, and is specifically designed to improve the resiliency of distributed neural networks.
  arXiv  Detail & Related papers  (2020-02-18T05:58:24Z)
• Noisy Machines: Understanding Noisy Neural Networks and Enhancing Robustness to Analog Hardware Errors Using Distillation [12.30062870698165]
  We show how a noisy neural network has reduced learning capacity as a result of loss of mutual information between its input and output. We propose using knowledge distillation combined with noise injection during training to achieve more noise robust networks. Our method achieves models with as much as two times greater noise tolerance compared with the previous best attempts.
  arXiv  Detail & Related papers  (2020-01-14T18:59:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.