Sparse by Rule: Probability-Based N:M Pruning for Spiking Neural Networks

• URL: http://arxiv.org/abs/2511.12097v1
• Date: Sat, 15 Nov 2025 08:23:20 GMT
• Title: Sparse by Rule: Probability-Based N:M Pruning for Spiking Neural Networks
• Authors: Shuhan Ye, Yi Yu, Qixin Zhang, Chenqi Kong, Qiangqiang Wu, Xudong Jiang, Dacheng Tao,
• Abstract summary: Spiking neural networks (SNNs) promise energy-efficient intelligence via event-driven, sparse computation, but deeper architectures inflate parameters and computational cost, hindering their edge deployment.<n>Recent progress in SNN pruning helps alleviate this burden, yet existing efforts fall into only two families: emphunstructured pruning, which attains high sparsity but is difficult to accelerate on general hardware, and emphstructured pruning, which eases deployment but lack flexibility and often degrades accuracy at matched sparsity.<n>We introduce bfSpikeNM, the first SNN
• Score: 66.61171793101872
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Brain-inspired Spiking neural networks (SNNs) promise energy-efficient intelligence via event-driven, sparse computation, but deeper architectures inflate parameters and computational cost, hindering their edge deployment. Recent progress in SNN pruning helps alleviate this burden, yet existing efforts fall into only two families: \emph{unstructured} pruning, which attains high sparsity but is difficult to accelerate on general hardware, and \emph{structured} pruning, which eases deployment but lack flexibility and often degrades accuracy at matched sparsity. In this work, we introduce \textbf{SpikeNM}, the first SNN-oriented \emph{semi-structured} \(N{:}M\) pruning framework that learns sparse SNNs \emph{from scratch}, enforcing \emph{at most \(N\)} non-zeros per \(M\)-weight block. To avoid the combinatorial space complexity \(\sum_{k=1}^{N}\binom{M}{k}\) growing exponentially with \(M\), SpikeNM adopts an \(M\)-way basis-logit parameterization with a differentiable top-\(k\) sampler, \emph{linearizing} per-block complexity to \(\mathcal O(M)\) and enabling more aggressive sparsification. Further inspired by neuroscience, we propose \emph{eligibility-inspired distillation} (EID), which converts temporally accumulated credits into block-wise soft targets to align mask probabilities with spiking dynamics, reducing sampling variance and stabilizing search under high sparsity. Experiments show that at \(2{:}4\) sparsity, SpikeNM maintains and even with gains across main-stream datasets, while yielding hardware-amenable patterns that complement intrinsic spike sparsity.

Related papers

• Time Is All It Takes: Spike-Retiming Attacks on Event-Driven Spiking Neural Networks [87.16809558673403]
  Spiking neural networks (SNNs) compute with discrete spikes and exploit temporal structure.<n>We study a timing-only adversary that retimes existing spikes while preserving spike counts and amplitudes in event-driven SNNs.
  arXiv  Detail & Related papers  (2026-02-03T09:06:53Z)
• S$^2$NN: Sub-bit Spiking Neural Networks [53.08060832135342]
  Spiking Neural Networks (SNNs) offer an energy-efficient paradigm for machine intelligence.<n>Despite recent advances in binary SNNs, the storage and computational demands remain substantial for large-scale networks.<n>We propose Sub-bit Spiking Neural Networks (S$2$NNs) that represent weights with less than one bit.
  arXiv  Detail & Related papers  (2025-09-29T04:17:44Z)
• Barycentric Neural Networks and Length-Weighted Persistent Entropy Loss: A Green Geometric and Topological Framework for Function Approximation [0.0]
  We introduce the Barycentric Neural Network (BNN), a compact shallow architecture that encodes both structure and parameters through a fixed set of base points and their associated barycentric coordinates.<n>We show that the BNN enables the exact representation of continuous piecewise linear functions (CPLFs), ensuring strict continuity across segments.<n>Our approach integrates the BNN with a loss function based on LWPE to optimize the base points that define the BNN, rather than its internal parameters.
  arXiv  Detail & Related papers  (2025-09-08T13:47:21Z)
• Efficient Link Prediction via GNN Layers Induced by Negative Sampling [86.87385758192566]
  Graph neural networks (GNNs) for link prediction can loosely be divided into two broad categories.<n>We propose a novel GNN architecture whereby the emphforward pass explicitly depends on emphboth positive (as is typical) and negative (unique to our approach) edges.<n>This is achieved by recasting the embeddings themselves as minimizers of a forward-pass-specific energy function that favors separation of positive and negative samples.
  arXiv  Detail & Related papers  (2023-10-14T07:02:54Z)
• SUBP: Soft Uniform Block Pruning for 1xN Sparse CNNs Multithreading Acceleration [16.846777341261436]
  The study of sparsity in Convolutional Neural Networks (CNNs) has become widespread to compress and accelerate models in environments with limited resources. Recent work requires selecting and fine-tuning 1$times$N sparse weights based on dense pre-trained weights. This paper proposes a novel emphtextbfSoft textbfUniform textbfBlock textbfPruning (SUBP) approach to train a uniform 1$times$N sparse structured network from scratch.
  arXiv  Detail & Related papers  (2023-10-10T00:22:27Z)
• Towards Zero Memory Footprint Spiking Neural Network Training [7.4331790419913455]
  Spiking Neural Networks (SNNs) process information using discrete-time events known as spikes rather than continuous values. In this paper, we introduce an innovative framework characterized by a remarkably low memory footprint. Our design is able to achieve a $mathbf58.65times$ reduction in memory usage compared to the current SNN node.
  arXiv  Detail & Related papers  (2023-08-16T19:49:24Z)
• Spatial Re-parameterization for N:M Sparsity [99.38650135086931]
  SpRe employs the spatial-sparsity distribution of unstructured sparsity by assigning an extra branch in conjunction with the original N:M branch at training time.<n>SpRe has achieved a commendable feat by matching the performance of N:M sparsity methods with state-of-the-art unstructured sparsity methods.
  arXiv  Detail & Related papers  (2023-06-09T01:11:50Z)
• Exploring Tradeoffs in Spiking Neural Networks [0.0]
  Spiking Neural Networks (SNNs) have emerged as a promising alternative to traditional Deep Neural Networks for low-power computing. We show that relaxing the spike constraint provides higher performance while also benefiting from faster convergence, similar sparsity, comparable prediction latency, and better robustness to noise compared to TTFS SNNs.
  arXiv  Detail & Related papers  (2022-12-15T20:18:14Z)
• Efficient Learning of Mesh-Based Physical Simulation with BSMS-GNN [36.73790892258642]
  Bi-stride pools nodes on every other frontier of breadth-first search. One-MP scheme per level and non-parametrized pooling, resembling U-Nets, significantly reduces computational costs.
  arXiv  Detail & Related papers  (2022-10-05T21:45:16Z)
• A temporally and spatially local spike-based backpropagation algorithm to enable training in hardware [0.0]
  Spiking Neural Networks (SNNs) have emerged as a hardware efficient architecture for classification tasks. There have been several attempts to adopt the powerful backpropagation (BP) technique used in non-spiking artificial neural networks (ANNs)
  arXiv  Detail & Related papers  (2022-07-20T08:57:53Z)
• Learning N:M Fine-grained Structured Sparse Neural Networks From Scratch [75.69506249886622]
  Sparsity in Deep Neural Networks (DNNs) has been widely studied to compress and accelerate the models on resource-constrained environments. In this paper, we are the first to study training from scratch an N:M fine-grained structured sparse network.
  arXiv  Detail & Related papers  (2021-02-08T05:55:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.