Causal pieces: analysing and improving spiking neural networks piece by piece

• URL: http://arxiv.org/abs/2504.14015v1
• Date: Fri, 18 Apr 2025 18:07:33 GMT
• Title: Causal pieces: analysing and improving spiking neural networks piece by piece
• Authors: Dominik Dold, Philipp Christian Petersen,
• Abstract summary: We introduce a novel concept for spiking neural networks (SNNs) derived from the idea of "linear pieces"<n>We prove that the input domain of SNNs decomposes into distinct causal regions where its output spike times are locally Lipschitz continuous.<n>The number of such regions - which we call "causal pieces" - is a measure of the approximation capabilities of SNNs.
• Score: 2.255961793913651
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce a novel concept for spiking neural networks (SNNs) derived from the idea of "linear pieces" used to analyse the expressiveness and trainability of artificial neural networks (ANNs). We prove that the input domain of SNNs decomposes into distinct causal regions where its output spike times are locally Lipschitz continuous with respect to the input spike times and network parameters. The number of such regions - which we call "causal pieces" - is a measure of the approximation capabilities of SNNs. In particular, we demonstrate in simulation that parameter initialisations which yield a high number of causal pieces on the training set strongly correlate with SNN training success. Moreover, we find that feedforward SNNs with purely positive weights exhibit a surprisingly high number of causal pieces, allowing them to achieve competitive performance levels on benchmark tasks. We believe that causal pieces are not only a powerful and principled tool for improving SNNs, but might also open up new ways of comparing SNNs and ANNs in the future.

Related papers

• Enhancing Adversarial Robustness in SNNs with Sparse Gradients [46.15229142258264]
  Spiking Neural Networks (SNNs) have attracted great attention for their energy-efficient operations and biologically inspired structures. Existing techniques, whether adapted from ANNs or specifically designed for SNNs, exhibit limitations in training SNNs or defending against strong attacks. We propose a novel approach to enhance the robustness of SNNs through gradient sparsity regularization.
  arXiv  Detail & Related papers  (2024-05-30T05:39:27Z)
• Inherent Redundancy in Spiking Neural Networks [24.114844269113746]
  Spiking Networks (SNNs) are a promising energy-efficient alternative to conventional artificial neural networks. In this work, we focus on three key questions regarding inherent redundancy in SNNs. We propose an Advance Attention (ASA) module to harness SNNs' redundancy.
  arXiv  Detail & Related papers  (2023-08-16T08:58:25Z)
• Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study [55.12108376616355]
  The study on NTK has been devoted to typical neural network architectures, but is incomplete for neural networks with Hadamard products (NNs-Hp) In this work, we derive the finite-width-K formulation for a special class of NNs-Hp, i.e., neural networks. We prove their equivalence to the kernel regression predictor with the associated NTK, which expands the application scope of NTK.
  arXiv  Detail & Related papers  (2022-09-16T06:36:06Z)
• On the Intrinsic Structures of Spiking Neural Networks [66.57589494713515]
  Recent years have emerged a surge of interest in SNNs owing to their remarkable potential to handle time-dependent and event-driven data. There has been a dearth of comprehensive studies examining the impact of intrinsic structures within spiking computations. This work delves deep into the intrinsic structures of SNNs, by elucidating their influence on the expressivity of SNNs.
  arXiv  Detail & Related papers  (2022-06-21T09:42:30Z)
• Examining the Robustness of Spiking Neural Networks on Non-ideal Memristive Crossbars [4.184276171116354]
  Spiking Neural Networks (SNNs) have emerged as the low-power alternative to Artificial Neural Networks (ANNs) We study the effect of crossbar non-idealities and intrinsicity on the performance of SNNs.
  arXiv  Detail & Related papers  (2022-06-20T07:07:41Z)
• Training High-Performance Low-Latency Spiking Neural Networks by Differentiation on Spike Representation [70.75043144299168]
  Spiking Neural Network (SNN) is a promising energy-efficient AI model when implemented on neuromorphic hardware. It is a challenge to efficiently train SNNs due to their non-differentiability. We propose the Differentiation on Spike Representation (DSR) method, which could achieve high performance.
  arXiv  Detail & Related papers  (2022-05-01T12:44:49Z)
• Deep Learning in Spiking Phasor Neural Networks [0.6767885381740952]
  Spiking Neural Networks (SNNs) have attracted the attention of the deep learning community for use in low-latency, low-power neuromorphic hardware. In this paper, we introduce Spiking Phasor Neural Networks (SPNNs) SPNNs are based on complex-valued Deep Neural Networks (DNNs), representing phases by spike times.
  arXiv  Detail & Related papers  (2022-04-01T15:06:15Z)
• Comparative Analysis of Interval Reachability for Robust Implicit and Feedforward Neural Networks [64.23331120621118]
  We use interval reachability analysis to obtain robustness guarantees for implicit neural networks (INNs) INNs are a class of implicit learning models that use implicit equations as layers. We show that our approach performs at least as well as, and generally better than, applying state-of-the-art interval bound propagation methods to INNs.
  arXiv  Detail & Related papers  (2022-04-01T03:31:27Z)
• Spiking Neural Networks for Visual Place Recognition via Weighted Neuronal Assignments [24.754429120321365]
  Spiking neural networks (SNNs) offer both compelling potential advantages, including energy efficiency and low latencies. One promising area for high performance SNNs is template matching and image recognition. This research introduces the first high performance SNN for the Visual Place Recognition (VPR) task.
  arXiv  Detail & Related papers  (2021-09-14T05:40:40Z)
• Pruning of Deep Spiking Neural Networks through Gradient Rewiring [41.64961999525415]
  Spiking Neural Networks (SNNs) have been attached great importance due to their biological plausibility and high energy-efficiency on neuromorphic chips. Most existing methods directly apply pruning approaches in artificial neural networks (ANNs) to SNNs, which ignore the difference between ANNs and SNNs. We propose gradient rewiring (Grad R), a joint learning algorithm of connectivity and weight for SNNs, that enables us to seamlessly optimize network structure without retrain.
  arXiv  Detail & Related papers  (2021-05-11T10:05:53Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.