ReverB-SNN: Reversing Bit of the Weight and Activation for Spiking Neural Networks

• URL: http://arxiv.org/abs/2506.07720v1
• Date: Mon, 09 Jun 2025 13:02:03 GMT
• Title: ReverB-SNN: Reversing Bit of the Weight and Activation for Spiking Neural Networks
• Authors: Yufei Guo, Yuhan Zhang, Zhou Jie, Xiaode Liu, Xin Tong, Yuanpei Chen, Weihang Peng, Zhe Ma,
• Abstract summary: Spiking Neural Network (SNN), a biologically inspired neural network infrastructure, has garnered significant attention recently.<n>We advocate reversing the bit of the weight and activation for SNNs, called textbfReverB-SNN.<n>This preserves the event-driven and multiplication-free advantages of standard SNNs while enhancing the information capacity of activations.
• Score: 22.66861050525175
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The Spiking Neural Network (SNN), a biologically inspired neural network infrastructure, has garnered significant attention recently. SNNs utilize binary spike activations for efficient information transmission, replacing multiplications with additions, thereby enhancing energy efficiency. However, binary spike activation maps often fail to capture sufficient data information, resulting in reduced accuracy. To address this challenge, we advocate reversing the bit of the weight and activation for SNNs, called \textbf{ReverB-SNN}, inspired by recent findings that highlight greater accuracy degradation from quantizing activations compared to weights. Specifically, our method employs real-valued spike activations alongside binary weights in SNNs. This preserves the event-driven and multiplication-free advantages of standard SNNs while enhancing the information capacity of activations. Additionally, we introduce a trainable factor within binary weights to adaptively learn suitable weight amplitudes during training, thereby increasing network capacity. To maintain efficiency akin to vanilla \textbf{ReverB-SNN}, our trainable binary weight SNNs are converted back to standard form using a re-parameterization technique during inference. Extensive experiments across various network architectures and datasets, both static and dynamic, demonstrate that our approach consistently outperforms state-of-the-art methods.

Related papers

• Deep activity propagation via weight initialization in spiking neural networks [10.69085409825724]
  Spiking Neural Networks (SNNs) offer bio-inspired advantages such as sparsity and ultra-low power consumption.<n>Deep SNNs process and transmit information by quantizing the real-valued membrane potentials into binary spikes.<n>We show theoretically that, unlike standard approaches, this method enables the propagation of activity in deep SNNs without loss of spikes.
  arXiv  Detail & Related papers  (2024-10-01T11:02:34Z)
• Recurrent Bilinear Optimization for Binary Neural Networks [58.972212365275595]
  BNNs neglect the intrinsic bilinear relationship of real-valued weights and scale factors. Our work is the first attempt to optimize BNNs from the bilinear perspective. We obtain robust RBONNs, which show impressive performance over state-of-the-art BNNs on various models and datasets.
  arXiv  Detail & Related papers  (2022-09-04T06:45:33Z)
• Examining the Robustness of Spiking Neural Networks on Non-ideal Memristive Crossbars [4.184276171116354]
  Spiking Neural Networks (SNNs) have emerged as the low-power alternative to Artificial Neural Networks (ANNs) We study the effect of crossbar non-idealities and intrinsicity on the performance of SNNs.
  arXiv  Detail & Related papers  (2022-06-20T07:07:41Z)
• Can Deep Neural Networks be Converted to Ultra Low-Latency Spiking Neural Networks? [3.2108350580418166]
  Spiking neural networks (SNNs) operate via binary spikes distributed over time. SOTA training strategies for SNNs involve conversion from a non-spiking deep neural network (DNN) We propose a new training algorithm that accurately captures these distributions, minimizing the error between the DNN and converted SNN.
  arXiv  Detail & Related papers  (2021-12-22T18:47:45Z)
• Spatial-Temporal-Fusion BNN: Variational Bayesian Feature Layer [77.78479877473899]
  We design a spatial-temporal-fusion BNN for efficiently scaling BNNs to large models. Compared to vanilla BNNs, our approach can greatly reduce the training time and the number of parameters, which contributes to scale BNNs efficiently.
  arXiv  Detail & Related papers  (2021-12-12T17:13:14Z)
• Sub-bit Neural Networks: Learning to Compress and Accelerate Binary Neural Networks [72.81092567651395]
  Sub-bit Neural Networks (SNNs) are a new type of binary quantization design tailored to compress and accelerate BNNs. SNNs are trained with a kernel-aware optimization framework, which exploits binary quantization in the fine-grained convolutional kernel space. Experiments on visual recognition benchmarks and the hardware deployment on FPGA validate the great potentials of SNNs.
  arXiv  Detail & Related papers  (2021-10-18T11:30:29Z)
• Dynamic Binary Neural Network by learning channel-wise thresholds [9.432747511001246]
  We propose a dynamic BNN (DyBNN) incorporating dynamic learnable channel-wise thresholds of Sign function and shift parameters of PReLU. The DyBNN based on two backbones of ReActNet (MobileNetV1 and ResNet18) achieve 71.2% and 67.4% top1-accuracy on ImageNet dataset.
  arXiv  Detail & Related papers  (2021-10-08T17:41:36Z)
• ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training [68.63354877166756]
  ActNN is a memory-efficient training framework that stores randomly quantized activations for back propagation. ActNN reduces the memory footprint of the activation by 12x, and it enables training with a 6.6x to 14x larger batch size.
  arXiv  Detail & Related papers  (2021-04-29T05:50:54Z)
• BiSNN: Training Spiking Neural Networks with Binary Weights via Bayesian Learning [37.376989855065545]
  Spiking Neural Networks (SNNs) are biologically inspired, dynamic, event-driven models that enhance energy efficiency. An SNN model is introduced that combines the benefits of temporally sparse binary activations and of binary weights. Experiments validate the performance loss with respect to full-precision implementations.
  arXiv  Detail & Related papers  (2020-12-15T14:06:36Z)
• Deep Time Delay Neural Network for Speech Enhancement with Full Data Learning [60.20150317299749]
  This paper proposes a deep time delay neural network (TDNN) for speech enhancement with full data learning. To make full use of the training data, we propose a full data learning method for speech enhancement.
  arXiv  Detail & Related papers  (2020-11-11T06:32:37Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.