Generalization Bounds of Spiking Neural Networks via Rademacher Complexity

• URL: http://arxiv.org/abs/2605.02927v1
• Date: Sun, 26 Apr 2026 14:24:48 GMT
• Title: Generalization Bounds of Spiking Neural Networks via Rademacher Complexity
• Abstract summary: Spiking Neural Networks (SNNs) have garnered increasing attention as one of bio-inspired models due to their great potential in neuromorphic computing and computation.<n>Recent advances disclosed an excitation-dependent and architecture-related generalization bound such that the Rademacher complexity of SNNs with firing can be upper bounded by an exponential function.<n>We theoretically investigate the generalization bounds of SNNs with several integration-and-fire schemes via Rademacher complexity.
• Score: 58.338455812259724
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Spiking Neural Networks (SNNs) have garnered increasing attention as one of bio-inspired models due to their great potential in neuromorphic computing and sparse computation. Many practical algorithms and techniques have been developed; however, theoretical understandings of the generalization, that is, the extent to which SNNs perform well on unseen data, are far from clear. Recent advances disclosed an excitation-dependent and architecture-related generalization bound such that the Rademacher complexity of SNNs with stochastic firing can be upper bounded by an exponential function relative to the excitation probability and the architecture depth. In this paper, we theoretically investigate the generalization bounds of SNNs with several integration-and-fire schemes via Rademacher complexity. We recognize that the empirical Rademacher complexity of SNNs is close to the SNN configurations, which is exponential to the network depth and the maximum time duration of received spike sequences, superlinear and subquadratic to the network width, polynomial to the parameter norm, inverse-linear to the number of training samples, and independent of the computations within spiking neurons, achieving a more precise rate than conventional studies. Our theoretical results may support the scope of SNN theories and shed some insight into the development of SNNs.

Related papers

• Parameter efficient hybrid spiking-quantum convolutional neural network with surrogate gradient and quantum data-reupload [0.20999222360659606]
  Spiking Quantum Neural Network (SQNN) combines principles from spiking neural networks (SNNs) and quantum computing.<n>SQDR-CNN enables joint training of convolutional SNNs and quantum circuits within a single backpropagation framework.<n>We achieve 86% of the mean top-performing accuracy of the SOTA SNN baselines, yet uses only 0.5% of the smallest spiking model's parameters.
  arXiv  Detail & Related papers  (2025-12-03T15:43:33Z)
• Accuracy, Memory Efficiency and Generalization: A Comparative Study on Liquid Neural Networks and Recurrent Neural Networks [4.8039887662381995]
  Review aims to conduct a comparative analysis of liquid neural networks (LNNs) and traditional recurrent neural networks (RNNs)<n>Research findings reveal that LNN, as an emerging, biologically inspired, continuous-time dynamic neural network, demonstrates significant potential in handling noisy, non-stationary data.<n>Some LNN variants outperform traditional RNN in terms of parameter efficiency and computational speed.
  arXiv  Detail & Related papers  (2025-10-08T21:50:34Z)
• Flexible and Scalable Deep Dendritic Spiking Neural Networks with Multiple Nonlinear Branching [39.664692909673086]
  We propose the dendritic spiking neuron (DendSN) incorporating multiple dendritic branches with nonlinear dynamics.<n>Compared to the point spiking neurons, DendSN exhibits significantly higher expressivity.<n>Our work demonstrates the possibility of training bio-plausible dendritic SNNs with depths and scales comparable to traditional point SNNs.
  arXiv  Detail & Related papers  (2024-12-09T10:15:46Z)
• On Neural Networks as Infinite Tree-Structured Probabilistic Graphical Models [44.676210493587256]
  We propose an innovative solution by constructing infinite tree-structured PGMs that correspond exactly to neural networks.<n>Our research reveals that DNNs, during forward propagation, indeed perform approximations of PGM inference that are precise in this alternative PGM structure.
  arXiv  Detail & Related papers  (2023-05-27T21:32:28Z)
• A Hybrid Neural Coding Approach for Pattern Recognition with Spiking Neural Networks [53.31941519245432]
  Brain-inspired spiking neural networks (SNNs) have demonstrated promising capabilities in solving pattern recognition tasks. These SNNs are grounded on homogeneous neurons that utilize a uniform neural coding for information representation. In this study, we argue that SNN architectures should be holistically designed to incorporate heterogeneous coding schemes.
  arXiv  Detail & Related papers  (2023-05-26T02:52:12Z)
• Intrinsic and extrinsic deep learning on manifolds [2.207988653560308]
  intrinsic deep neural networks (iDNNs) incorporate the underlying intrinsic geometry of manifold via exponential and log maps. We prove that the empirical risk of the empirical risk minimizers (ERM) of eDNNs and iDNNs converge in optimal rates. The eDNNs framework is simple and easy to compute, while the iDNNs framework is accurate and fast converging.
  arXiv  Detail & Related papers  (2023-02-16T22:10:38Z)
• Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study [55.12108376616355]
  The study on NTK has been devoted to typical neural network architectures, but is incomplete for neural networks with Hadamard products (NNs-Hp) In this work, we derive the finite-width-K formulation for a special class of NNs-Hp, i.e., neural networks. We prove their equivalence to the kernel regression predictor with the associated NTK, which expands the application scope of NTK.
  arXiv  Detail & Related papers  (2022-09-16T06:36:06Z)
• On the Intrinsic Structures of Spiking Neural Networks [66.57589494713515]
  Recent years have emerged a surge of interest in SNNs owing to their remarkable potential to handle time-dependent and event-driven data. There has been a dearth of comprehensive studies examining the impact of intrinsic structures within spiking computations. This work delves deep into the intrinsic structures of SNNs, by elucidating their influence on the expressivity of SNNs.
  arXiv  Detail & Related papers  (2022-06-21T09:42:30Z)
• Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysis [94.64007376939735]
  We theoretically characterize the impact of connectivity patterns on the convergence of deep neural networks (DNNs) under gradient descent training. We show that by a simple filtration on "unpromising" connectivity patterns, we can trim down the number of models to evaluate.
  arXiv  Detail & Related papers  (2022-05-11T17:43:54Z)
• Universal approximation property of invertible neural networks [76.95927093274392]
  Invertible neural networks (INNs) are neural network architectures with invertibility by design. Thanks to their invertibility and the tractability of Jacobian, INNs have various machine learning applications such as probabilistic modeling, generative modeling, and representation learning.
  arXiv  Detail & Related papers  (2022-04-15T10:45:26Z)
• Extended critical regimes of deep neural networks [0.0]
  We show that heavy-tailed weights enable the emergence of an extended critical regime without fine-tuning parameters. In this extended critical regime, DNNs exhibit rich and complex propagation dynamics across layers. We provide a theoretical guide for the design of efficient neural architectures.
  arXiv  Detail & Related papers  (2022-03-24T10:15:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.