EchoSpike Predictive Plasticity: An Online Local Learning Rule for Spiking Neural Networks

• URL: http://arxiv.org/abs/2405.13976v2
• Date: Sun, 26 May 2024 15:20:51 GMT
• Title: EchoSpike Predictive Plasticity: An Online Local Learning Rule for Spiking Neural Networks
• Authors: Lars Graf, Zhe Su, Giacomo Indiveri,
• Abstract summary: Spiking Neural Networks (SNNs) are attractive due to their potential in applications requiring low power and memory. "EchoSpike Predictive Plasticity" (ESPP) learning rule is a pioneering online local learning rule. ESPP represents a significant advancement in developing biologically plausible self-supervised learning models for neuromorphic computing at the edge.
• Score: 4.644628459389789
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The drive to develop artificial neural networks that efficiently utilize resources has generated significant interest in bio-inspired Spiking Neural Networks (SNNs). These networks are particularly attractive due to their potential in applications requiring low power and memory. This potential is further enhanced by the ability to perform online local learning, enabling them to adapt to dynamic environments. This requires the model to be adaptive in a self-supervised manner. While self-supervised learning has seen great success in many deep learning domains, its application for online local learning in multi-layer SNNs remains underexplored. In this paper, we introduce the "EchoSpike Predictive Plasticity" (ESPP) learning rule, a pioneering online local learning rule designed to leverage hierarchical temporal dynamics in SNNs through predictive and contrastive coding. We validate the effectiveness of this approach using benchmark datasets, demonstrating that it performs on par with current state-of-the-art supervised learning rules. The temporal and spatial locality of ESPP makes it particularly well-suited for low-cost neuromorphic processors, representing a significant advancement in developing biologically plausible self-supervised learning models for neuromorphic computing at the edge.

Related papers

• Fully Spiking Actor Network with Intra-layer Connections for Reinforcement Learning [51.386945803485084]
  We focus on the task where the agent needs to learn multi-dimensional deterministic policies to control. Most existing spike-based RL methods take the firing rate as the output of SNNs, and convert it to represent continuous action space (i.e., the deterministic policy) through a fully-connected layer. To develop a fully spiking actor network without any floating-point matrix operations, we draw inspiration from the non-spiking interneurons found in insects.
  arXiv  Detail & Related papers  (2024-01-09T07:31:34Z)
• SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence [51.6943465041708]
  Spiking neural networks (SNNs) aim to realize brain-inspired intelligence on neuromorphic chips with high energy efficiency. We contribute a full-stack toolkit for pre-processing neuromorphic datasets, building deep SNNs, optimizing their parameters, and deploying SNNs on neuromorphic chips.
  arXiv  Detail & Related papers  (2023-10-25T13:15:17Z)
• How neural networks learn to classify chaotic time series [77.34726150561087]
  We study the inner workings of neural networks trained to classify regular-versus-chaotic time series. We find that the relation between input periodicity and activation periodicity is key for the performance of LKCNN models.
  arXiv  Detail & Related papers  (2023-06-04T08:53:27Z)
• PC-SNN: Supervised Learning with Local Hebbian Synaptic Plasticity based on Predictive Coding in Spiking Neural Networks [1.6172800007896282]
  We propose a novel learning algorithm inspired by predictive coding theory. We show that it can perform supervised learning fully autonomously and successfully as the backprop. This method achieves a favorable performance compared to the state-of-the-art multi-layer SNNs.
  arXiv  Detail & Related papers  (2022-11-24T09:56:02Z)
• An Unsupervised STDP-based Spiking Neural Network Inspired By Biologically Plausible Learning Rules and Connections [10.188771327458651]
  Spike-timing-dependent plasticity (STDP) is a general learning rule in the brain, but spiking neural networks (SNNs) trained with STDP alone is inefficient and perform poorly. We design an adaptive synaptic filter and introduce the adaptive spiking threshold to enrich the representation ability of SNNs. Our model achieves the current state-of-the-art performance of unsupervised STDP-based SNNs in the MNIST and FashionMNIST datasets.
  arXiv  Detail & Related papers  (2022-07-06T14:53:32Z)
• Ensemble plasticity and network adaptability in SNNs [0.726437825413781]
  Artificial Spiking Neural Networks (ASNNs) promise greater information processing efficiency because of discrete event-based (i.e., spike) computation. We introduce a novel ensemble learning method based on entropy and network activation, operated exclusively using spiking activity. It was discovered that pruning lower spike-rate neuron clusters resulted in increased generalization or a predictable decline in performance.
  arXiv  Detail & Related papers  (2022-03-11T01:14:51Z)
• In-Hardware Learning of Multilayer Spiking Neural Networks on a Neuromorphic Processor [6.816315761266531]
  This work presents a spike-based backpropagation algorithm with biological plausible local update rules and adapts it to fit the constraint in a neuromorphic hardware. The algorithm is implemented on Intel Loihi chip enabling low power in- hardware supervised online learning of multilayered SNNs for mobile applications.
  arXiv  Detail & Related papers  (2021-05-08T09:22:21Z)
• Online Spatio-Temporal Learning in Deep Neural Networks [1.6624384368855523]
  Online learning has recently regained the attention of the research community, focusing on approaches that approximate BPTT or on biologically-plausible schemes applied to SNNs. Here we present an alternative perspective that is based on a clear separation of spatial and temporal gradient components. We derive from first principles a novel online learning algorithm for deep SNNs, called online spiking-temporal learning (OSTL) For shallow networks, OSTL is gradient-equivalent to BPTT enabling for the first time online training of SNNs with BPTT-equivalent gradients. In addition, the proposed formulation unveils a class of SNN architectures
  arXiv  Detail & Related papers  (2020-07-24T18:10:18Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)
• Recurrent Neural Network Learning of Performance and Intrinsic Population Dynamics from Sparse Neural Data [77.92736596690297]
  We introduce a novel training strategy that allows learning not only the input-output behavior of an RNN but also its internal network dynamics. We test the proposed method by training an RNN to simultaneously reproduce internal dynamics and output signals of a physiologically-inspired neural model. Remarkably, we show that the reproduction of the internal dynamics is successful even when the training algorithm relies on the activities of a small subset of neurons.
  arXiv  Detail & Related papers  (2020-05-05T14:16:54Z)
• Rectified Linear Postsynaptic Potential Function for Backpropagation in Deep Spiking Neural Networks [55.0627904986664]
  Spiking Neural Networks (SNNs) usetemporal spike patterns to represent and transmit information, which is not only biologically realistic but also suitable for ultra-low-power event-driven neuromorphic implementation. This paper investigates the contribution of spike timing dynamics to information encoding, synaptic plasticity and decision making, providing a new perspective to design of future DeepSNNs and neuromorphic hardware systems.
  arXiv  Detail & Related papers  (2020-03-26T11:13:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.