PMSN: A Parallel Multi-compartment Spiking Neuron for Multi-scale Temporal Processing

• URL: http://arxiv.org/abs/2408.14917v1
• Date: Tue, 27 Aug 2024 09:47:46 GMT
• Title: PMSN: A Parallel Multi-compartment Spiking Neuron for Multi-scale Temporal Processing
• Authors: Xinyi Chen, Jibin Wu, Chenxiang Ma, Yinsong Yan, Yujie Wu, Kay Chen Tan,
• Abstract summary: Spiking Neural Networks (SNNs) hold great potential to realize brain-inspired, energy-efficient computational systems. We present a novel spiking neuron model called Parallel Multi-compartment Spiking Neuron (PMSN) PMSN emulates biological neurons by incorporating multiple interacting substructures and allows for flexible adjustment of the substructure counts.
• Score: 22.1268533721837
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Spiking Neural Networks (SNNs) hold great potential to realize brain-inspired, energy-efficient computational systems. However, current SNNs still fall short in terms of multi-scale temporal processing compared to their biological counterparts. This limitation has resulted in poor performance in many pattern recognition tasks with information that varies across different timescales. To address this issue, we put forward a novel spiking neuron model called Parallel Multi-compartment Spiking Neuron (PMSN). The PMSN emulates biological neurons by incorporating multiple interacting substructures and allows for flexible adjustment of the substructure counts to effectively represent temporal information across diverse timescales. Additionally, to address the computational burden associated with the increased complexity of the proposed model, we introduce two parallelization techniques that decouple the temporal dependencies of neuronal updates, enabling parallelized training across different time steps. Our experimental results on a wide range of pattern recognition tasks demonstrate the superiority of PMSN. It outperforms other state-of-the-art spiking neuron models in terms of its temporal processing capacity, training speed, and computation cost. Specifically, compared with the commonly used Leaky Integrate-and-Fire neuron, PMSN offers a simulation acceleration of over 10 $\times$ and a 30 % improvement in accuracy on Sequential CIFAR10 dataset, while maintaining comparable computational cost.

Related papers

• Spiking Neural Networks for Temporal Processing: Status Quo and Future Prospects [41.8742357294068]
  Spiking Neural Networks (SNNs) excel in handling data with high efficiency, owing to their rich neuronal dynamics and sparse activity patterns. Given the recent surge in the development of SNNs, there is an urgent need for a comprehensive evaluation of their temporal processing capabilities.
  arXiv  Detail & Related papers  (2025-02-13T16:17:57Z)
• Channel-wise Parallelizable Spiking Neuron with Multiplication-free Dynamics and Large Temporal Receptive Fields [32.349167886062105]
  Spiking Neural Networks (SNNs) are distinguished from Artificial Neural Networks (ANNs) for their sophisticated neuronal dynamics and sparse binary activations (spikes) inspired by the biological neural system. Traditional neuron models use iterative step-by-step dynamics, resulting in serial computation and slow training speed of SNNs. Recent parallelizable spiking neuron models have been proposed to fully utilize the massive parallel computing ability of graphics processing units to accelerate the training of SNNs.
  arXiv  Detail & Related papers  (2025-01-24T13:44:08Z)
• Deep-Unrolling Multidimensional Harmonic Retrieval Algorithms on Neuromorphic Hardware [78.17783007774295]
  This paper explores the potential of conversion-based neuromorphic algorithms for highly accurate and energy-efficient single-snapshot multidimensional harmonic retrieval. A novel method for converting the complex-valued convolutional layers and activations into spiking neural networks (SNNs) is developed. The converted SNNs achieve almost five-fold power efficiency at moderate performance loss compared to the original CNNs.
  arXiv  Detail & Related papers  (2024-12-05T09:41:33Z)
• Time-independent Spiking Neuron via Membrane Potential Estimation for Efficient Spiking Neural Networks [4.142699381024752]
  computational inefficiency of spiking neural networks (SNNs) is primarily due to the sequential updates of membrane potential. We propose Membrane Potential Estimation Parallel Spiking Neurons (MPE-PSN), a parallel computation method for spiking neurons. Our approach exhibits promise for enhancing computational efficiency, particularly under conditions of elevated neuron density.
  arXiv  Detail & Related papers  (2024-09-08T05:14:22Z)
• Temporal Spiking Neural Networks with Synaptic Delay for Graph Reasoning [91.29876772547348]
  Spiking neural networks (SNNs) are investigated as biologically inspired models of neural computation. This paper reveals that SNNs, when amalgamated with synaptic delay and temporal coding, are proficient in executing (knowledge) graph reasoning.
  arXiv  Detail & Related papers  (2024-05-27T05:53:30Z)
• GRSN: Gated Recurrent Spiking Neurons for POMDPs and MARL [28.948871773551854]
  Spiking neural networks (SNNs) are widely applied in various fields due to their energy-efficient and fast-inference capabilities. In current spiking reinforcement learning (SRL) algorithms, the simulation results of multiple time steps can only correspond to a single-step decision in RL. We propose a novel temporal alignment paradigm (TAP) that leverages the single-step update of spiking neurons to accumulate historical state information in RL.
  arXiv  Detail & Related papers  (2024-04-24T02:20:50Z)
• Long Short-term Memory with Two-Compartment Spiking Neuron [64.02161577259426]
  We propose a novel biologically inspired Long Short-Term Memory Leaky Integrate-and-Fire spiking neuron model, dubbed LSTM-LIF. Our experimental results, on a diverse range of temporal classification tasks, demonstrate superior temporal classification capability, rapid training convergence, strong network generalizability, and high energy efficiency of the proposed LSTM-LIF model. This work, therefore, opens up a myriad of opportunities for resolving challenging temporal processing tasks on emerging neuromorphic computing machines.
  arXiv  Detail & Related papers  (2023-07-14T08:51:03Z)
• The Expressive Leaky Memory Neuron: an Efficient and Expressive Phenomenological Neuron Model Can Solve Long-Horizon Tasks [64.08042492426992]
  We introduce the Expressive Memory (ELM) neuron model, a biologically inspired model of a cortical neuron. Our ELM neuron can accurately match the aforementioned input-output relationship with under ten thousand trainable parameters. We evaluate it on various tasks with demanding temporal structures, including the Long Range Arena (LRA) datasets.
  arXiv  Detail & Related papers  (2023-06-14T13:34:13Z)
• Intelligence Processing Units Accelerate Neuromorphic Learning [52.952192990802345]
  Spiking neural networks (SNNs) have achieved orders of magnitude improvement in terms of energy consumption and latency. We present an IPU-optimized release of our custom SNN Python package, snnTorch.
  arXiv  Detail & Related papers  (2022-11-19T15:44:08Z)
• Training Feedback Spiking Neural Networks by Implicit Differentiation on the Equilibrium State [66.2457134675891]
  Spiking neural networks (SNNs) are brain-inspired models that enable energy-efficient implementation on neuromorphic hardware. Most existing methods imitate the backpropagation framework and feedforward architectures for artificial neural networks. We propose a novel training method that does not rely on the exact reverse of the forward computation.
  arXiv  Detail & Related papers  (2021-09-29T07:46:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.