Optimal ANN-SNN Conversion with Group Neurons

• URL: http://arxiv.org/abs/2402.19061v1
• Date: Thu, 29 Feb 2024 11:41:12 GMT
• Title: Optimal ANN-SNN Conversion with Group Neurons
• Authors: Liuzhenghao Lv, Wei Fang, Li Yuan, Yonghong Tian
• Abstract summary: Spiking Neural Networks (SNNs) have emerged as a promising third generation of neural networks. The lack of effective learning algorithms remains a challenge for SNNs. We introduce a novel type of neuron called Group Neurons (GNs)
• Score: 39.14228133571838
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Spiking Neural Networks (SNNs) have emerged as a promising third generation of neural networks, offering unique characteristics such as binary outputs, high sparsity, and biological plausibility. However, the lack of effective learning algorithms remains a challenge for SNNs. For instance, while converting artificial neural networks (ANNs) to SNNs circumvents the need for direct training of SNNs, it encounters issues related to conversion errors and high inference time delays. In order to reduce or even eliminate conversion errors while decreasing inference time-steps, we have introduced a novel type of neuron called Group Neurons (GNs). One GN is composed of multiple Integrate-and-Fire (IF) neurons as members, and its neural dynamics are meticulously designed. Based on GNs, we have optimized the traditional ANN-SNN conversion framework. Specifically, we replace the IF neurons in the SNNs obtained by the traditional conversion framework with GNs. The resulting SNNs, which utilize GNs, are capable of achieving accuracy levels comparable to ANNs even within extremely short inference time-steps. The experiments on CIFAR10, CIFAR100, and ImageNet datasets demonstrate the superiority of the proposed methods in terms of both inference accuracy and latency. Code is available at https://github.com/Lyu6PosHao/ANN2SNN_GN.

Related papers

• High-performance deep spiking neural networks with 0.3 spikes per neuron [9.01407445068455]
  It is hard to train biologically-inspired spiking neural networks (SNNs) than artificial neural networks (ANNs) We show that training deep SNN models achieves the exact same performance as that of ANNs. Our SNN accomplishes high-performance classification with less than 0.3 spikes per neuron, lending itself for an energy-efficient implementation.
  arXiv  Detail & Related papers  (2023-06-14T21:01:35Z)
• Fast-SNN: Fast Spiking Neural Network by Converting Quantized ANN [38.18008827711246]
  Spiking neural networks (SNNs) have shown advantages in computation and energy efficiency. It remains a challenge to train deep SNNs due to the discrete spike function. This paper proposes Fast-SNN that achieves high performance with low latency.
  arXiv  Detail & Related papers  (2023-05-31T14:04:41Z)
• A Hybrid Neural Coding Approach for Pattern Recognition with Spiking Neural Networks [53.31941519245432]
  Brain-inspired spiking neural networks (SNNs) have demonstrated promising capabilities in solving pattern recognition tasks. These SNNs are grounded on homogeneous neurons that utilize a uniform neural coding for information representation. In this study, we argue that SNN architectures should be holistically designed to incorporate heterogeneous coding schemes.
  arXiv  Detail & Related papers  (2023-05-26T02:52:12Z)
• SEENN: Towards Temporal Spiking Early-Exit Neural Networks [26.405775809170308]
  Spiking Neural Networks (SNNs) have recently become more popular as a biologically plausible substitute for traditional Artificial Neural Networks (ANNs) We study a fine-grained adjustment of the number of timesteps in SNNs. By dynamically adjusting the number of timesteps, our SEENN achieves a remarkable reduction in the average number of timesteps during inference.
  arXiv  Detail & Related papers  (2023-04-02T15:57:09Z)
• Training High-Performance Low-Latency Spiking Neural Networks by Differentiation on Spike Representation [70.75043144299168]
  Spiking Neural Network (SNN) is a promising energy-efficient AI model when implemented on neuromorphic hardware. It is a challenge to efficiently train SNNs due to their non-differentiability. We propose the Differentiation on Spike Representation (DSR) method, which could achieve high performance.
  arXiv  Detail & Related papers  (2022-05-01T12:44:49Z)
• Sub-bit Neural Networks: Learning to Compress and Accelerate Binary Neural Networks [72.81092567651395]
  Sub-bit Neural Networks (SNNs) are a new type of binary quantization design tailored to compress and accelerate BNNs. SNNs are trained with a kernel-aware optimization framework, which exploits binary quantization in the fine-grained convolutional kernel space. Experiments on visual recognition benchmarks and the hardware deployment on FPGA validate the great potentials of SNNs.
  arXiv  Detail & Related papers  (2021-10-18T11:30:29Z)
• Beyond Classification: Directly Training Spiking Neural Networks for Semantic Segmentation [5.800785186389827]
  Spiking Neural Networks (SNNs) have emerged as the low-power alternative to Artificial Neural Networks (ANNs) In this paper, we explore the SNN applications beyond classification and present semantic segmentation networks configured with spiking neurons.
  arXiv  Detail & Related papers  (2021-10-14T21:53:03Z)
• Optimal Conversion of Conventional Artificial Neural Networks to Spiking Neural Networks [0.0]
  Spiking neural networks (SNNs) are biology-inspired artificial neural networks (ANNs) We propose a novel strategic pipeline that transfers the weights to the target SNN by combining threshold balance and soft-reset mechanisms. Our method is promising to get implanted onto embedded platforms with better support of SNNs with limited energy and memory.
  arXiv  Detail & Related papers  (2021-02-28T12:04:22Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)
• You Only Spike Once: Improving Energy-Efficient Neuromorphic Inference to ANN-Level Accuracy [51.861168222799186]
  Spiking Neural Networks (SNNs) are a type of neuromorphic, or brain-inspired network. SNNs are sparse, accessing very few weights, and typically only use addition operations instead of the more power-intensive multiply-and-accumulate operations. In this work, we aim to overcome the limitations of TTFS-encoded neuromorphic systems.
  arXiv  Detail & Related papers  (2020-06-03T15:55:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.