Temporal-adaptive Weight Quantization for Spiking Neural Networks

• URL: http://arxiv.org/abs/2511.17567v1
• Date: Fri, 14 Nov 2025 05:08:14 GMT
• Title: Temporal-adaptive Weight Quantization for Spiking Neural Networks
• Abstract summary: We propose Temporal-adaptive Weight Quantization (TaWQ), which incorporates weight quantization with temporal dynamics to adaptively allocate ultra-low-bit weights along the temporal dimension.<n>Our TaWQ maintains high energy efficiency (4.12M, 0.63mJ) while incurring a negligible quantization loss of only 0.22% on ImageNet.
• Score: 40.9339917499242
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Weight quantization in spiking neural networks (SNNs) could further reduce energy consumption. However, quantizing weights without sacrificing accuracy remains challenging. In this study, inspired by astrocyte-mediated synaptic modulation in the biological nervous systems, we propose Temporal-adaptive Weight Quantization (TaWQ), which incorporates weight quantization with temporal dynamics to adaptively allocate ultra-low-bit weights along the temporal dimension. Extensive experiments on static (e.g., ImageNet) and neuromorphic (e.g., CIFAR10-DVS) datasets demonstrate that our TaWQ maintains high energy efficiency (4.12M, 0.63mJ) while incurring a negligible quantization loss of only 0.22% on ImageNet.

Related papers

• Efficient and Robust Spiking Neural Networks for sEMG-Based Muscle Fatigue Detection [6.801129260713164]
  We propose an energy-efficient framework for muscle fatigue detection based on Spiking Neural Networks (SNNs)<n>We introduce a quantization-compatible robustness training scheme (SDH) that combines multiple regularization terms to improve computation under noisy conditions.<n>Results demonstrate the framework's strong potential for real-time deployment in low-power wearable systems.
  arXiv  Detail & Related papers  (2026-07-13T04:05:13Z)
• MD-SNN: Membrane Potential-aware Distillation on Quantized Spiking Neural Network [18.23285395499578]
  Spiking Neural Networks (SNNs) offer a promising and energy-efficient alternative to conventional neural networks.<n>SNNs face challenges regarding memory and computation due to complex-temporal dynamics.<n>We introduce Membrane-aware Distillation on quantized Spiking Neural Network (MD-SNN)
  arXiv  Detail & Related papers  (2025-12-04T04:27:19Z)
• DilateQuant: Accurate and Efficient Diffusion Quantization via Weight Dilation [9.612860507356702]
  Post-training quantization (PTQ) fails catastrophically at low-bit cases.<n>Quantization-aware training (QAT) is essential.<n>We propose a novel QAT framework for diffusion models, called DilateQuant.
  arXiv  Detail & Related papers  (2024-09-22T04:21:29Z)
• CTRQNets & LQNets: Continuous Time Recurrent and Liquid Quantum Neural Networks [76.53016529061821]
  Liquid Quantum Neural Network (LQNet) and Continuous Time Recurrent Quantum Neural Network (CTRQNet) developed.<n>LQNet and CTRQNet achieve accuracy increases as high as 40% on CIFAR 10 through binary classification.
  arXiv  Detail & Related papers  (2024-08-28T00:56:03Z)
• Quantum-limited stochastic optical neural networks operating at a few quanta per activation [6.12660587056967]
  Energy efficiency in computation is ultimately limited by noise, with quantum limits setting the fundamental noise floor.<n>We study optical neural networks where all layers except the last are operated in the limit that each neuron can be activated by just a single photon.<n>We show that by using a physics-based probabilistic model of the neuron activations in training, it is possible to perform accurate machine-learning inference in spite of the extremely high shot noise.
  arXiv  Detail & Related papers  (2023-07-28T17:59:46Z)
• Towards Neural Variational Monte Carlo That Scales Linearly with System Size [67.09349921751341]
  Quantum many-body problems are central to demystifying some exotic quantum phenomena, e.g., high-temperature superconductors. The combination of neural networks (NN) for representing quantum states, and the Variational Monte Carlo (VMC) algorithm, has been shown to be a promising method for solving such problems. We propose a NN architecture called Vector-Quantized Neural Quantum States (VQ-NQS) that utilizes vector-quantization techniques to leverage redundancies in the local-energy calculations of the VMC algorithm.
  arXiv  Detail & Related papers  (2022-12-21T19:00:04Z)
• BiTAT: Neural Network Binarization with Task-dependent Aggregated Transformation [116.26521375592759]
  Quantization aims to transform high-precision weights and activations of a given neural network into low-precision weights/activations for reduced memory usage and computation. Extreme quantization (1-bit weight/1-bit activations) of compactly-designed backbone architectures results in severe performance degeneration. This paper proposes a novel Quantization-Aware Training (QAT) method that can effectively alleviate performance degeneration.
  arXiv  Detail & Related papers  (2022-07-04T13:25:49Z)
• Momentum Diminishes the Effect of Spectral Bias in Physics-Informed Neural Networks [72.09574528342732]
  Physics-informed neural network (PINN) algorithms have shown promising results in solving a wide range of problems involving partial differential equations (PDEs) They often fail to converge to desirable solutions when the target function contains high-frequency features, due to a phenomenon known as spectral bias. In the present work, we exploit neural tangent kernels (NTKs) to investigate the training dynamics of PINNs evolving under gradient descent with momentum (SGDM)
  arXiv  Detail & Related papers  (2022-06-29T19:03:10Z)
• Overcoming Oscillations in Quantization-Aware Training [18.28657022169428]
  When training neural networks with simulated quantization, quantized weights can, rather unexpectedly, oscillate between two grid-points. We show that it can lead to a significant accuracy degradation due to wrongly estimated batch-normalization statistics. We propose two novel QAT algorithms to overcome oscillations during training: oscillation dampening and iterative weight freezing.
  arXiv  Detail & Related papers  (2022-03-21T16:07:42Z)
• DNN Quantization with Attention [5.72175302235089]
  We propose a training procedure that relaxes the low-bit quantization. The relaxation is achieved by using a learnable linear combination of high, medium and low-bit quantizations. In experiments, our approach outperforms other low-bit quantization techniques.
  arXiv  Detail & Related papers  (2021-03-24T16:24:59Z)
• Neural networks with late-phase weights [66.72777753269658]
  We show that the solutions found by SGD can be further improved by ensembling a subset of the weights in late stages of learning. At the end of learning, we obtain back a single model by taking a spatial average in weight space.
  arXiv  Detail & Related papers  (2020-07-25T13:23:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.