The Backpropagation Algorithm Implemented on Spiking Neuromorphic Hardware

• URL: http://arxiv.org/abs/2106.07030v1
• Date: Sun, 13 Jun 2021 15:56:40 GMT
• Title: The Backpropagation Algorithm Implemented on Spiking Neuromorphic Hardware
• Authors: Alpha Renner, Forrest Sheldon, Anatoly Zlotnik, Louis Tao, Andrew Sornborger
• Abstract summary: We present a neuromorphic, spiking backpropagation algorithm based on pulse-gated dynamical information coordination and processing. We demonstrate a proof-of-principle three-layer circuit that learns to classify digits from the MNIST dataset.
• Score: 4.3310896118860445
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The capabilities of natural neural systems have inspired new generations of machine learning algorithms as well as neuromorphic very large-scale integrated (VLSI) circuits capable of fast, low-power information processing. However, most modern machine learning algorithms are not neurophysiologically plausible and thus are not directly implementable in neuromorphic hardware. In particular, the workhorse of modern deep learning, the backpropagation algorithm, has proven difficult to translate to neuromorphic hardware. In this study, we present a neuromorphic, spiking backpropagation algorithm based on pulse-gated dynamical information coordination and processing, implemented on Intel's Loihi neuromorphic research processor. We demonstrate a proof-of-principle three-layer circuit that learns to classify digits from the MNIST dataset. This implementation shows a path for using massively parallel, low-power, low-latency neuromorphic processors in modern deep learning applications.

Related papers

• Analysis of Generalized Hebbian Learning Algorithm for Neuromorphic Hardware Using Spinnaker [0.0]
  We show the application of the Generalized Hebbian Algorithm (GHA) in large-scale neuromorphic platforms, specifically SpiNNaker. Our results demonstrate significant improvements in classification accuracy, showcasing the potential of biologically inspired learning algorithms.
  arXiv  Detail & Related papers  (2024-11-18T13:53:10Z)
• Contrastive Learning in Memristor-based Neuromorphic Systems [55.11642177631929]
  Spiking neural networks have become an important family of neuron-based models that sidestep many of the key limitations facing modern-day backpropagation-trained deep networks. In this work, we design and investigate a proof-of-concept instantiation of contrastive-signal-dependent plasticity (CSDP), a neuromorphic form of forward-forward-based, backpropagation-free learning.
  arXiv  Detail & Related papers  (2024-09-17T04:48:45Z)
• Neuromorphic Intermediate Representation: A Unified Instruction Set for Interoperable Brain-Inspired Computing [4.066607775161713]
  Spiking neural networks and neuromorphic hardware platforms that simulate neuronal dynamics are getting wide attention. Here, we establish a common reference frame for computations in digital neuromorphic systems. We demonstrate by reproducing three spiking neural network models of different complexity across 7 neuromorphic simulators and 4 digital hardware platforms.
  arXiv  Detail & Related papers  (2023-11-24T18:15:59Z)
• SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence [51.6943465041708]
  Spiking neural networks (SNNs) aim to realize brain-inspired intelligence on neuromorphic chips with high energy efficiency. We contribute a full-stack toolkit for pre-processing neuromorphic datasets, building deep SNNs, optimizing their parameters, and deploying SNNs on neuromorphic chips.
  arXiv  Detail & Related papers  (2023-10-25T13:15:17Z)
• Neuromorphic Artificial Intelligence Systems [58.1806704582023]
  Modern AI systems, based on von Neumann architecture and classical neural networks, have a number of fundamental limitations in comparison with the brain. This article discusses such limitations and the ways they can be mitigated. It presents an overview of currently available neuromorphic AI projects in which these limitations are overcome.
  arXiv  Detail & Related papers  (2022-05-25T20:16:05Z)
• Mapping and Validating a Point Neuron Model on Intel's Neuromorphic Hardware Loihi [77.34726150561087]
  We investigate the potential of Intel's fifth generation neuromorphic chip - Loihi' Loihi is based on the novel idea of Spiking Neural Networks (SNNs) emulating the neurons in the brain. We find that Loihi replicates classical simulations very efficiently and scales notably well in terms of both time and energy performance as the networks get larger.
  arXiv  Detail & Related papers  (2021-09-22T16:52:51Z)
• In-Hardware Learning of Multilayer Spiking Neural Networks on a Neuromorphic Processor [6.816315761266531]
  This work presents a spike-based backpropagation algorithm with biological plausible local update rules and adapts it to fit the constraint in a neuromorphic hardware. The algorithm is implemented on Intel Loihi chip enabling low power in- hardware supervised online learning of multilayered SNNs for mobile applications.
  arXiv  Detail & Related papers  (2021-05-08T09:22:21Z)
• Neuromorphic Algorithm-hardware Codesign for Temporal Pattern Learning [11.781094547718595]
  We derive an efficient training algorithm for Leaky Integrate and Fire neurons, which is capable of training a SNN to learn complex spatial temporal patterns. We have developed a CMOS circuit implementation for a memristor-based network of neuron and synapses which retains critical neural dynamics with reduced complexity.
  arXiv  Detail & Related papers  (2021-04-21T18:23:31Z)
• Brain-Inspired Learning on Neuromorphic Substrates [5.279475826661643]
  This article provides a mathematical framework for the design of practical online learning algorithms for neuromorphic substrates. Specifically, we show a direct connection between Real-Time Recurrent Learning (RTRL) and biologically plausible learning rules for training Spiking Neural Networks (SNNs) We motivate a sparse approximation based on block-diagonal Jacobians, which reduces the algorithm's computational complexity.
  arXiv  Detail & Related papers  (2020-10-22T17:56:59Z)
• Spiking Neural Networks Hardware Implementations and Challenges: a Survey [53.429871539789445]
  Spiking Neural Networks are cognitive algorithms mimicking neuron and synapse operational principles. We present the state of the art of hardware implementations of spiking neural networks. We discuss the strategies employed to leverage the characteristics of these event-driven algorithms at the hardware level.
  arXiv  Detail & Related papers  (2020-05-04T13:24:00Z)
• Rectified Linear Postsynaptic Potential Function for Backpropagation in Deep Spiking Neural Networks [55.0627904986664]
  Spiking Neural Networks (SNNs) usetemporal spike patterns to represent and transmit information, which is not only biologically realistic but also suitable for ultra-low-power event-driven neuromorphic implementation. This paper investigates the contribution of spike timing dynamics to information encoding, synaptic plasticity and decision making, providing a new perspective to design of future DeepSNNs and neuromorphic hardware systems.
  arXiv  Detail & Related papers  (2020-03-26T11:13:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.