Reconsidering the energy efficiency of spiking neural networks

• URL: http://arxiv.org/abs/2409.08290v2
• Date: Thu, 03 Jul 2025 10:37:52 GMT
• Title: Reconsidering the energy efficiency of spiking neural networks
• Authors: Zhanglu Yan, Zhenyu Bai, Weng-Fai Wong,
• Abstract summary: Spiking Neural Networks (SNNs) promise higher energy efficiency over conventional Quantized Artificial Neural Networks (QNNs)<n>This paper presents a rigorous re-evaluation of the true energy benefits of SNNs.
• Score: 4.37952937111446
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Spiking Neural Networks (SNNs) promise higher energy efficiency over conventional Quantized Artificial Neural Networks (QNNs) due to their event-driven, spike-based computation. However, prevailing energy evaluations often oversimplify, focusing on computational aspects while neglecting critical overheads like comprehensive data movement and memory access. Such simplifications can lead to misleading conclusions regarding the true energy benefits of SNNs. This paper presents a rigorous re-evaluation. We establish a fair baseline by mapping rate-encoded SNNs with $T$ timesteps to functionally equivalent QNNs with $\lceil \log_2(T+1) \rceil$ bits. This ensures both models have comparable representational capacities, as well has similar hardware requirement, enabling meaningful energy comparisons. We introduce a detailed analytical energy model encompassing core computation and data movement (sparse and dense activations, weights). Using this model, we systematically explore a wide parameter space, including intrinsic network characteristics ($T$, spike rate $s_r$, QNN sparsity $\gamma$, model size $N$, weight bit-level) and hardware characteristics (memory system and network-on-chip). Our analysis identifies specific operational regimes where SNNs genuinely offer superior energy efficiency. For example, under typical neuromorphic hardware conditions, SNNs with moderate time windows ($T \in [5,10]$) require an average spike rate ($s_r$) below 6.4% to outperform equivalent QNNs. These insights guide the design of genuinely energy-efficient neural network solutions.

Related papers

• Energy efficiency analysis of Spiking Neural Networks for space applications [43.91307921405309]
  Spiking Neural Networks (SNN) are highly attractive due to their theoretically superior energy efficiency.<n>This work presents a numerical analysis and comparison of different SNN techniques applied to scene classification for the EuroSAT dataset.
  arXiv  Detail & Related papers  (2025-05-16T16:29:50Z)
• LightSNN: Lightweight Architecture Search for Sparse and Accurate Spiking Neural Networks [1.0485739694839666]
  Spiking Neural Networks (SNNs) are highly regarded for their energy efficiency, inherent activation sparsity, and suitability for real-time processing in edge devices. Most current SNN methods adopt architectures resembling traditional artificial neural networks (ANNs) leading to suboptimal performance when applied to SNNs. We present LightSNN, a rapid and efficient Neural Network Architecture Search (NAS) technique specifically tailored for SNNs.
  arXiv  Detail & Related papers  (2025-03-27T16:38:13Z)
• Differential Coding for Training-Free ANN-to-SNN Conversion [45.70141988713627]
  Spiking Neural Networks (SNNs) exhibit significant potential due to their low energy consumption. converting Artificial Neural Networks (ANNs) to SNNs is an efficient way to achieve high-performance SNNs. This article introduces differential coding for ANN-to-SNN conversion, a novel coding scheme that reduces spike counts and energy consumption.
  arXiv  Detail & Related papers  (2025-03-01T02:17:35Z)
• GhostRNN: Reducing State Redundancy in RNN with Cheap Operations [66.14054138609355]
  We propose an efficient RNN architecture, GhostRNN, which reduces hidden state redundancy with cheap operations. Experiments on KWS and SE tasks demonstrate that the proposed GhostRNN significantly reduces the memory usage (40%) and computation cost while keeping performance similar.
  arXiv  Detail & Related papers  (2024-11-20T11:37:14Z)
• NeuroNAS: Enhancing Efficiency of Neuromorphic In-Memory Computing for Intelligent Mobile Agents through Hardware-Aware Spiking Neural Architecture Search [6.006032394972252]
  Spiking Neural Networks (SNNs) leverage event-based computation to enable ultra-low power/energy machine learning algorithms. NeuroNAS is a novel framework for developing energy-efficient neuromorphic IMC for intelligent mobile agents.
  arXiv  Detail & Related papers  (2024-06-30T09:51:58Z)
• Temporal Spiking Neural Networks with Synaptic Delay for Graph Reasoning [91.29876772547348]
  Spiking neural networks (SNNs) are investigated as biologically inspired models of neural computation. This paper reveals that SNNs, when amalgamated with synaptic delay and temporal coding, are proficient in executing (knowledge) graph reasoning.
  arXiv  Detail & Related papers  (2024-05-27T05:53:30Z)
• SparrowSNN: A Hardware/software Co-design for Energy Efficient ECG Classification [7.030659971563306]
  spiking neural networks (SNNs) are well-known for their energy efficiency. sparrowSNN achieves a state-of-the-art accuracy of 98.29% for SNNs, with energy consumption of 31.39nJ per inference and power usage of 6.1uW.
  arXiv  Detail & Related papers  (2024-05-06T10:30:05Z)
• Continuous Spiking Graph Neural Networks [43.28609498855841]
  Continuous graph neural networks (CGNNs) have garnered significant attention due to their ability to generalize existing discrete graph neural networks (GNNs) We introduce the high-order structure of COS-GNN, which utilizes the second-order ODE for spiking representation and continuous propagation. We provide the theoretical proof that COS-GNN effectively mitigates the issues of exploding and vanishing gradients, enabling us to capture long-range dependencies between nodes.
  arXiv  Detail & Related papers  (2024-04-02T12:36:40Z)
• Bayesian Inference Accelerator for Spiking Neural Networks [3.145754107337963]
  spiking neural networks (SNNs) have the potential to reduce computational area and power. In this work, we demonstrate an optimization framework for developing and implementing efficient Bayesian SNNs in hardware. We demonstrate accuracies comparable to Bayesian binary networks with full-precision Bernoulli parameters, while requiring up to $25times$ less spikes.
  arXiv  Detail & Related papers  (2024-01-27T16:27:19Z)
• LitE-SNN: Designing Lightweight and Efficient Spiking Neural Network through Spatial-Temporal Compressive Network Search and Joint Optimization [48.41286573672824]
  Spiking Neural Networks (SNNs) mimic the information-processing mechanisms of the human brain and are highly energy-efficient. We propose a new approach named LitE-SNN that incorporates both spatial and temporal compression into the automated network design process.
  arXiv  Detail & Related papers  (2024-01-26T05:23:11Z)
• Pursing the Sparse Limitation of Spiking Deep Learning Structures [42.334835610250714]
  Spiking Neural Networks (SNNs) are garnering increased attention for their superior computation and energy efficiency. We introduce an innovative algorithm capable of simultaneously identifying both weight and patch-level winning tickets. We demonstrate that our spiking lottery ticket achieves comparable or superior performance even when the model structure is extremely sparse.
  arXiv  Detail & Related papers  (2023-11-18T17:00:40Z)
• Is Conventional SNN Really Efficient? A Perspective from Network Quantization [7.04833025737147]
  Spiking Neural Networks (SNNs) have been widely praised for their high energy efficiency and immense potential. However, comprehensive research that critically contrasts and correlates SNNs with quantized Artificial Neural Networks (ANNs) remains scant. This paper introduces a unified perspective, illustrating that the time steps in SNNs and quantized bit-widths of activation values present analogous representations.
  arXiv  Detail & Related papers  (2023-11-17T09:48:22Z)
• Are SNNs Truly Energy-efficient? $-$ A Hardware Perspective [7.539212567508529]
  Spiking Neural Networks (SNNs) have gained attention for their energy-efficient machine learning capabilities. This work studies two hardware benchmarking platforms for large-scale SNN inference, namely SATA and SpikeSim.
  arXiv  Detail & Related papers  (2023-09-06T22:23:22Z)
• Adaptive-SpikeNet: Event-based Optical Flow Estimation using Spiking Neural Networks with Learnable Neuronal Dynamics [6.309365332210523]
  Spiking Neural Networks (SNNs) with their neuro-inspired event-driven processing can efficiently handle asynchronous data. We propose an adaptive fully-spiking framework with learnable neuronal dynamics to alleviate the spike vanishing problem. Our experiments on datasets show an average reduction of 13% in average endpoint error (AEE) compared to state-of-the-art ANNs.
  arXiv  Detail & Related papers  (2022-09-21T21:17:56Z)
• Training High-Performance Low-Latency Spiking Neural Networks by Differentiation on Spike Representation [70.75043144299168]
  Spiking Neural Network (SNN) is a promising energy-efficient AI model when implemented on neuromorphic hardware. It is a challenge to efficiently train SNNs due to their non-differentiability. We propose the Differentiation on Spike Representation (DSR) method, which could achieve high performance.
  arXiv  Detail & Related papers  (2022-05-01T12:44:49Z)
• Weightless Neural Networks for Efficient Edge Inference [1.7882696915798877]
  Weightless Neural Networks (WNNs) are a class of machine learning model which use table lookups to perform inference. We propose a novel WNN architecture, BTHOWeN, with key algorithmic and architectural improvements over prior work. BTHOWeN targets the large and growing edge computing sector by providing superior latency and energy efficiency.
  arXiv  Detail & Related papers  (2022-03-03T01:46:05Z)
• S$^2$NN: Time Step Reduction of Spiking Surrogate Gradients for Training Energy Efficient Single-Step Neural Networks [0.40145248246551063]
  We propose a single-step neural network (S$2$NN) with low computational cost and high precision. The proposed S$2$NN processes the information between hidden layers by spikes as SNNs. It has no temporal dimension so that there is no latency within training and inference phases as BNNs.
  arXiv  Detail & Related papers  (2022-01-26T11:31:21Z)
• Keys to Accurate Feature Extraction Using Residual Spiking Neural Networks [1.101002667958165]
  Spiking neural networks (SNNs) have become an interesting alternative to conventional artificial neural networks (ANNs) We present a study on the key components of modern spiking architectures. We design a spiking version of the successful residual network (ResNet) architecture and test different components and training strategies on it.
  arXiv  Detail & Related papers  (2021-11-10T21:29:19Z)
• Neural network relief: a pruning algorithm based on neural activity [47.57448823030151]
  We propose a simple importance-score metric that deactivates unimportant connections. We achieve comparable performance for LeNet architectures on MNIST. The algorithm is not designed to minimize FLOPs when considering current hardware and software implementations.
  arXiv  Detail & Related papers  (2021-09-22T15:33:49Z)
• DTNN: Energy-efficient Inference with Dendrite Tree Inspired Neural Networks for Edge Vision Applications [2.1800759000607024]
  We propose Dendrite-Tree based Neural Network (DTNN) for energy-efficient inference with table lookup operations enabled by activation quantization. DTNN achieved significant energy saving (19.4X and 64.9X improvement on ResNet-18 and VGG-11 with ImageNet, respectively) with negligible loss of accuracy.
  arXiv  Detail & Related papers  (2021-05-25T11:44:12Z)
• ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training [68.63354877166756]
  ActNN is a memory-efficient training framework that stores randomly quantized activations for back propagation. ActNN reduces the memory footprint of the activation by 12x, and it enables training with a 6.6x to 14x larger batch size.
  arXiv  Detail & Related papers  (2021-04-29T05:50:54Z)
• Toward Trainability of Quantum Neural Networks [87.04438831673063]
  Quantum Neural Networks (QNNs) have been proposed as generalizations of classical neural networks to achieve the quantum speed-up. Serious bottlenecks exist for training QNNs due to the vanishing with gradient rate exponential to the input qubit number. We show that QNNs with tree tensor and step controlled structures for the application of binary classification. Simulations show faster convergent rates and better accuracy compared to QNNs with random structures.
  arXiv  Detail & Related papers  (2020-11-12T08:32:04Z)
• SmartExchange: Trading Higher-cost Memory Storage/Access for Lower-cost Computation [97.78417228445883]
  We present SmartExchange, an algorithm- hardware co-design framework for energy-efficient inference of deep neural networks (DNNs) We develop a novel algorithm to enforce a specially favorable DNN weight structure, where each layerwise weight matrix can be stored as the product of a small basis matrix and a large sparse coefficient matrix whose non-zero elements are all power-of-2. We further design a dedicated accelerator to fully utilize the SmartExchange-enforced weights to improve both energy efficiency and latency performance.
  arXiv  Detail & Related papers  (2020-05-07T12:12:49Z)
• Widening and Squeezing: Towards Accurate and Efficient QNNs [125.172220129257]
  Quantization neural networks (QNNs) are very attractive to the industry because their extremely cheap calculation and storage overhead, but their performance is still worse than that of networks with full-precision parameters. Most of existing methods aim to enhance performance of QNNs especially binary neural networks by exploiting more effective training techniques. We address this problem by projecting features in original full-precision networks to high-dimensional quantization features.
  arXiv  Detail & Related papers  (2020-02-03T04:11:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.