AutoSNN: Towards Energy-Efficient Spiking Neural Networks

• URL: http://arxiv.org/abs/2201.12738v1
• Date: Sun, 30 Jan 2022 06:12:59 GMT
• Title: AutoSNN: Towards Energy-Efficient Spiking Neural Networks
• Authors: Byunggook Na, Jisoo Mok, Seongsik Park, Dongjin Lee, Hyeokjun Choe, Sungroh Yoon
• Abstract summary: Spiking neural networks (SNNs) mimic information transmission in the brain. Most previous studies have focused solely on training methods, and the effect of architecture has rarely been studied. We propose a spike-aware neural architecture search framework called AutoSNN.
• Score: 26.288681480713695
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Spiking neural networks (SNNs) that mimic information transmission in the brain can energy-efficiently process spatio-temporal information through discrete and sparse spikes, thereby receiving considerable attention. To improve accuracy and energy efficiency of SNNs, most previous studies have focused solely on training methods, and the effect of architecture has rarely been studied. We investigate the design choices used in the previous studies in terms of the accuracy and number of spikes and figure out that they are not best-suited for SNNs. To further improve the accuracy and reduce the spikes generated by SNNs, we propose a spike-aware neural architecture search framework called AutoSNN. We define a search space consisting of architectures without undesirable design choices. To enable the spike-aware architecture search, we introduce a fitness that considers both the accuracy and number of spikes. AutoSNN successfully searches for SNN architectures that outperform hand-crafted SNNs in accuracy and energy efficiency. We thoroughly demonstrate the effectiveness of AutoSNN on various datasets including neuromorphic datasets.

Related papers

• NAS-BNN: Neural Architecture Search for Binary Neural Networks [55.058512316210056]
  We propose a novel neural architecture search scheme for binary neural networks, named NAS-BNN. Our discovered binary model family outperforms previous BNNs for a wide range of operations (OPs) from 20M to 200M. In addition, we validate the transferability of these searched BNNs on the object detection task, and our binary detectors with the searched BNNs achieve a novel state-of-the-art result, e.g., 31.6% mAP with 370M OPs, on MS dataset.
  arXiv  Detail & Related papers  (2024-08-28T02:17:58Z)
• SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence [51.6943465041708]
  Spiking neural networks (SNNs) aim to realize brain-inspired intelligence on neuromorphic chips with high energy efficiency. We contribute a full-stack toolkit for pre-processing neuromorphic datasets, building deep SNNs, optimizing their parameters, and deploying SNNs on neuromorphic chips.
  arXiv  Detail & Related papers  (2023-10-25T13:15:17Z)
• Inherent Redundancy in Spiking Neural Networks [24.114844269113746]
  Spiking Networks (SNNs) are a promising energy-efficient alternative to conventional artificial neural networks. In this work, we focus on three key questions regarding inherent redundancy in SNNs. We propose an Advance Attention (ASA) module to harness SNNs' redundancy.
  arXiv  Detail & Related papers  (2023-08-16T08:58:25Z)
• Deep Directly-Trained Spiking Neural Networks for Object Detection [20.594942840081757]
  EMS-YOLO is a novel directly-trained SNN framework for object detection. We design a full-spike residual block, EMS-ResNet, which can effectively extend the depth of the directly-trained SNN with low power consumption. It is shown that our model could achieve comparable performance to the ANN with the same architecture while consuming 5.83 times less energy.
  arXiv  Detail & Related papers  (2023-07-21T08:10:26Z)
• A Hybrid Neural Coding Approach for Pattern Recognition with Spiking Neural Networks [53.31941519245432]
  Brain-inspired spiking neural networks (SNNs) have demonstrated promising capabilities in solving pattern recognition tasks. These SNNs are grounded on homogeneous neurons that utilize a uniform neural coding for information representation. In this study, we argue that SNN architectures should be holistically designed to incorporate heterogeneous coding schemes.
  arXiv  Detail & Related papers  (2023-05-26T02:52:12Z)
• Training High-Performance Low-Latency Spiking Neural Networks by Differentiation on Spike Representation [70.75043144299168]
  Spiking Neural Network (SNN) is a promising energy-efficient AI model when implemented on neuromorphic hardware. It is a challenge to efficiently train SNNs due to their non-differentiability. We propose the Differentiation on Spike Representation (DSR) method, which could achieve high performance.
  arXiv  Detail & Related papers  (2022-05-01T12:44:49Z)
• Toward Robust Spiking Neural Network Against Adversarial Perturbation [22.56553160359798]
  spiking neural networks (SNNs) are deployed increasingly in real-world efficiency critical applications. Researchers have already demonstrated an SNN can be attacked with adversarial examples. To the best of our knowledge, this is the first analysis on robust training of SNNs.
  arXiv  Detail & Related papers  (2022-04-12T21:26:49Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)
• An Efficient Spiking Neural Network for Recognizing Gestures with a DVS Camera on the Loihi Neuromorphic Processor [12.118084418840152]
  Spiking Neural Networks (SNNs) have come under the spotlight for machine learning based applications. We show our methodology for the design of an SNN that achieves nearly the same accuracy results as its corresponding Deep Neural Networks (DNNs) Our SNN achieves 89.64% classification accuracy and occupies only 37 Loihi cores.
  arXiv  Detail & Related papers  (2020-05-16T17:00:10Z)
• Rectified Linear Postsynaptic Potential Function for Backpropagation in Deep Spiking Neural Networks [55.0627904986664]
  Spiking Neural Networks (SNNs) usetemporal spike patterns to represent and transmit information, which is not only biologically realistic but also suitable for ultra-low-power event-driven neuromorphic implementation. This paper investigates the contribution of spike timing dynamics to information encoding, synaptic plasticity and decision making, providing a new perspective to design of future DeepSNNs and neuromorphic hardware systems.
  arXiv  Detail & Related papers  (2020-03-26T11:13:07Z)
• SiamSNN: Siamese Spiking Neural Networks for Energy-Efficient Object Tracking [20.595208488431766]
  SiamSNN is the first deep SNN tracker that achieves short latency and low precision loss on the visual object tracking benchmarks OTB2013, VOT2016, and GOT-10k. SiamSNN notably achieves low energy consumption and real-time on Neuromorphic chip TrueNorth.
  arXiv  Detail & Related papers  (2020-03-17T08:49:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.