Scalable Training of Continuous-Time Spiking Neural Networks with Differentiable Spike-Time Discretization

• URL: http://arxiv.org/abs/2607.14672v1
• Date: Thu, 16 Jul 2026 07:38:24 GMT
• Title: Scalable Training of Continuous-Time Spiking Neural Networks with Differentiable Spike-Time Discretization
• Abstract summary: We introduce a memory-efficient training framework for continuous-time spiking neural networks (SNNs)<n>Differentiable spike-time discretization maps irregular presynaptic spikes onto differentiable weighted events at fixed time points.<n>In dense LIF layers, DSTD reduced peak memory consumption by up to approximately 100-fold and training time by up to 20-fold compared with exact spike-time computation.
• Score: 1.489944824637277
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Continuous-time spiking neural networks (SNNs) provide an event-driven framework for temporal computation, computational neuroscience, and neuromorphic hardware. However, training deep continuous-time SNNs is severely constrained by the memory required for exact spike-time computation, which evaluates and retains candidate firing times over intervals determined by presynaptic spike ordering. Here we introduce a memory-efficient training framework based on differentiable spike-time discretization (DSTD) for leaky integrate-and-fire neurons with general membrane and synaptic time constants. DSTD maps irregular presynaptic spikes onto differentiable weighted events at fixed time points, replacing the input-dependent candidate dimension with $M$ fixed time intervals while accurately approximating continuous-time membrane-potential dynamics. This reduces candidate-related activation memory from $O(N_{\mathrm{out}}N_{\mathrm{in}})$ to $O(N_{\mathrm{out}}M)$ in the case of time-to-first-spike (TTFS) coding, where $N_{\mathrm{in}}$ and $N_{\mathrm{out}}$ denote the numbers of presynaptic and postsynaptic neurons, respectively. We further introduce synfire-chain-inspired temporal regularization that organizes layer-wise firing windows, mitigates dead-neuron failures, and enables pipeline-like processing. In dense LIF layers, DSTD reduced peak memory consumption by up to approximately 100-fold and training time by up to approximately 20-fold compared with exact spike-time computation. Together, these methods allowed us to train 9-layer convolutional SNNs on CIFAR-10 and 20-layer convolutional SNNs on Fashion-MNIST on a single GPU.

Related papers

• LongSpike: Fractional Order Spiking State Space Models for Efficient Long Sequence Learning [53.855185131868446]
  Spiking Neural Networks (SNNs) are well-regarded for their biological plausibility and energy efficiency in processing sequential data.<n>In this work, we propose LongSpike, a novel SNN framework that integrates fractional-order State-Space Modeling.<n>By extending traditional integer-order SSMs to the fractional-calculus regime, LongSpike enables the hierarchical integration of neuronal dynamics with long-memory kernels.
  arXiv  Detail & Related papers  (2026-06-11T04:54:07Z)
• Pretraining Recurrent Networks without Recurrence [28.905819079755762]
  Training recurrent neural networks (RNNs) requires assigning credit across long sequences of computations.<n>We propose Supervised Memory Training (SMT), a method for training nonlinear RNNs that sidesteps recurrent credit propagation entirely.
  arXiv  Detail & Related papers  (2026-06-04T17:57:33Z)
• Bullet Trains: Parallelizing Training of Temporally Precise Spiking Neural Networks [0.7653446907404775]
  Continuous-time, event-native spiking neural networks (SNNs) operate strictly on spike events.<n>We address two challenges to practical, end-to-end trainable event-based SNN systems.<n>First, we use parallel associative scans to consume multiple input spikes at once, yielding up to 44x speedups over sequential simulation.<n>Second, we implement differentiable spike-time solvers that compute spike times to machine precision without discrete-time approximations or restrictive analytic assumptions.
  arXiv  Detail & Related papers  (2026-02-27T16:13:55Z)
• Time Is All It Takes: Spike-Retiming Attacks on Event-Driven Spiking Neural Networks [87.16809558673403]
  Spiking neural networks (SNNs) compute with discrete spikes and exploit temporal structure.<n>We study a timing-only adversary that retimes existing spikes while preserving spike counts and amplitudes in event-driven SNNs.
  arXiv  Detail & Related papers  (2026-02-03T09:06:53Z)
• Fractional Spike Differential Equations Neural Network with Efficient Adjoint Parameters Training [63.3991315762955]
  Spiking Neural Networks (SNNs) draw inspiration from biological neurons to create realistic models for brain-like computation.<n>Most existing SNNs assume a single time constant for neuronal membrane voltage dynamics, modeled by first-order ordinary differential equations (ODEs) with Markovian characteristics.<n>We propose the Fractional SPIKE Differential Equation neural network (fspikeDE), which captures long-term dependencies in membrane voltage and spike trains through fractional-order dynamics.
  arXiv  Detail & Related papers  (2025-07-22T18:20:56Z)
• Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity [63.56009745101597]
  Spiking Neural Networks (SNNs) often suffer from high time complexity $O(T)$ due to the sequential processing of $T$ spikes.<n>We propose a novel Fixed-point Parallel Training (FPT) method to accelerate SNN training without modifying the network architecture.
  arXiv  Detail & Related papers  (2025-06-10T13:27:27Z)
• PRES: Toward Scalable Memory-Based Dynamic Graph Neural Networks [22.47336262812308]
  Memory-based Dynamic Graph Neural Networks (MDGNNs) are a family of dynamic graph neural networks that leverage a memory module to extract, distill, and long-term temporal dependencies. This paper studies the efficient training of MDGNNs at scale, focusing on the temporal discontinuity in training MDGNNs with large temporal batch sizes.
  arXiv  Detail & Related papers  (2024-02-06T01:34:56Z)
• Accelerating SNN Training with Stochastic Parallelizable Spiking Neurons [1.7056768055368383]
  Spiking neural networks (SNN) are able to learn features while using less energy, especially on neuromorphic hardware. Most widely used neuron in deep learning is the temporal and Fire (LIF) neuron.
  arXiv  Detail & Related papers  (2023-06-22T04:25:27Z)
• Timing-Based Backpropagation in Spiking Neural Networks Without Single-Spike Restrictions [2.8360662552057323]
  We propose a novel backpropagation algorithm for training spiking neural networks (SNNs) It encodes information in the relative multiple spike timing of individual neurons without single-spike restrictions.
  arXiv  Detail & Related papers  (2022-11-29T11:38:33Z)
• Intelligence Processing Units Accelerate Neuromorphic Learning [52.952192990802345]
  Spiking neural networks (SNNs) have achieved orders of magnitude improvement in terms of energy consumption and latency. We present an IPU-optimized release of our custom SNN Python package, snnTorch.
  arXiv  Detail & Related papers  (2022-11-19T15:44:08Z)
• Training Feedback Spiking Neural Networks by Implicit Differentiation on the Equilibrium State [66.2457134675891]
  Spiking neural networks (SNNs) are brain-inspired models that enable energy-efficient implementation on neuromorphic hardware. Most existing methods imitate the backpropagation framework and feedforward architectures for artificial neural networks. We propose a novel training method that does not rely on the exact reverse of the forward computation.
  arXiv  Detail & Related papers  (2021-09-29T07:46:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.