ParaRevSNN: A Parallel Reversible Spiking Neural Network for Efficient Training and Inference

• URL: http://arxiv.org/abs/2508.01223v1
• Date: Sat, 02 Aug 2025 06:40:59 GMT
• Title: ParaRevSNN: A Parallel Reversible Spiking Neural Network for Efficient Training and Inference
• Authors: Changqing Xu, Guoqing Sun, Yi Liu, Xinfang Liao, Yintang Yang,
• Abstract summary: Reversible Spiking Neural Networks (RevSNNs) enable memory-efficient training by reconstructing forward activations during backpropagation.<n>RevSNNs suffer from high latency due to strictly sequential computation.<n>We propose ParaRevSNN, a parallel reversible SNN architecture that decouples sequential dependencies between reversible blocks.
• Score: 4.174294693108078
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reversible Spiking Neural Networks (RevSNNs) enable memory-efficient training by reconstructing forward activations during backpropagation, but suffer from high latency due to strictly sequential computation. To overcome this limitation, we propose ParaRevSNN, a parallel reversible SNN architecture that decouples sequential dependencies between reversible blocks while preserving reversibility. This design enables inter-block parallelism, significantly accelerating training and inference while retaining the memory-saving benefits of reversibility. Experiments on CIFAR10, CIFAR100, CIFAR10-DVS, and DVS128 Gesture demonstrate that ParaRevSNN matches or exceeds the accuracy of standard RevSNNs, while reducing training time by up to 35.2\% and inference time to 18.15\%, making it well-suited for deployment in resource-constrained scenarios.

Related papers

• Fast Training of Recurrent Neural Networks with Stationary State Feedbacks [48.22082789438538]
  Recurrent neural networks (RNNs) have recently demonstrated strong performance and faster inference than Transformers.<n>We propose a novel method that replaces BPTT with a fixed gradient feedback mechanism.
  arXiv  Detail & Related papers  (2025-03-29T14:45:52Z)
• Never Reset Again: A Mathematical Framework for Continual Inference in Recurrent Neural Networks [2.1838661321884443]
  Recurrent Neural Networks (RNNs) are widely used for sequential processing but face limitations with continual inference due to state saturation.<n>We propose an adaptive loss function that eliminates the need for resets during inference while preserving high accuracy over extended sequences.
  arXiv  Detail & Related papers  (2024-12-20T15:24:28Z)
• Memory-Efficient Reversible Spiking Neural Networks [8.05761813203348]
  Spiking neural networks (SNNs) are potential competitors to artificial neural networks (ANNs) SNNs require much more memory than ANNs, which impedes the training of deeper SNN models. We propose the reversible spiking neural network to reduce the memory cost of intermediate activations and membrane potentials during training.
  arXiv  Detail & Related papers  (2023-12-13T06:39:49Z)
• Towards Zero Memory Footprint Spiking Neural Network Training [7.4331790419913455]
  Spiking Neural Networks (SNNs) process information using discrete-time events known as spikes rather than continuous values. In this paper, we introduce an innovative framework characterized by a remarkably low memory footprint. Our design is able to achieve a $mathbf58.65times$ reduction in memory usage compared to the current SNN node.
  arXiv  Detail & Related papers  (2023-08-16T19:49:24Z)
• Resurrecting Recurrent Neural Networks for Long Sequences [45.800920421868625]
  Recurrent Neural Networks (RNNs) offer fast inference on long sequences but are hard to optimize and slow to train. Deep state-space models (SSMs) have recently been shown to perform remarkably well on long sequence modeling tasks. We show that careful design of deep RNNs using standard signal propagation arguments can recover the impressive performance of deep SSMs on long-range reasoning tasks.
  arXiv  Detail & Related papers  (2023-03-11T08:53:11Z)
• Online Training Through Time for Spiking Neural Networks [66.7744060103562]
  Spiking neural networks (SNNs) are promising brain-inspired energy-efficient models. Recent progress in training methods has enabled successful deep SNNs on large-scale tasks with low latency. We propose online training through time (OTTT) for SNNs, which is derived from BPTT to enable forward-in-time learning.
  arXiv  Detail & Related papers  (2022-10-09T07:47:56Z)
• Training High-Performance Low-Latency Spiking Neural Networks by Differentiation on Spike Representation [70.75043144299168]
  Spiking Neural Network (SNN) is a promising energy-efficient AI model when implemented on neuromorphic hardware. It is a challenge to efficiently train SNNs due to their non-differentiability. We propose the Differentiation on Spike Representation (DSR) method, which could achieve high performance.
  arXiv  Detail & Related papers  (2022-05-01T12:44:49Z)
• Effect of Architectures and Training Methods on the Performance of Learned Video Frame Prediction [10.404162481860634]
  Experimental results show that the residual FCNN architecture performs the best in terms of peak signal to noise ratio (PSNR) at the expense of higher training and test (inference) computational complexity. The CRNN can be trained stably and very efficiently using the stateful truncated backpropagation through time procedure.
  arXiv  Detail & Related papers  (2020-08-13T20:45:28Z)
• Distillation Guided Residual Learning for Binary Convolutional Neural Networks [83.6169936912264]
  It is challenging to bridge the performance gap between Binary CNN (BCNN) and Floating point CNN (FCNN) We observe that, this performance gap leads to substantial residuals between intermediate feature maps of BCNN and FCNN. To minimize the performance gap, we enforce BCNN to produce similar intermediate feature maps with the ones of FCNN. This training strategy, i.e., optimizing each binary convolutional block with block-wise distillation loss derived from FCNN, leads to a more effective optimization to BCNN.
  arXiv  Detail & Related papers  (2020-07-10T07:55:39Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.