Three factor delay learning rules for spiking neural networks

• URL: http://arxiv.org/abs/2601.00668v1
• Date: Fri, 02 Jan 2026 12:28:53 GMT
• Title: Three factor delay learning rules for spiking neural networks
• Authors: Luke Vassallo, Nima Taherinejad,
• Abstract summary: We introduce synaptic and axonal delays to integrate leaky and fire (LIF)-based feedforward and recurrent SNNs.<n>We propose three-constrained learning rules to simultaneously learn delay parameters online.<n>Our findings benefit the design of power and area-constrained neuromorphic processors by enabling on-device learning and lowering memory requirements.
• Score: 0.42970700836450487
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Spiking Neural Networks (SNNs) are dynamical systems that operate on spatiotemporal data, yet their learnable parameters are often limited to synaptic weights, contributing little to temporal pattern recognition. Learnable parameters that delay spike times can improve classification performance in temporal tasks, but existing methods rely on large networks and offline learning, making them unsuitable for real-time operation in resource-constrained environments. In this paper, we introduce synaptic and axonal delays to leaky integrate and fire (LIF)-based feedforward and recurrent SNNs, and propose three-factor learning rules to simultaneously learn delay parameters online. We employ a smooth Gaussian surrogate to approximate spike derivatives exclusively for the eligibility trace calculation, and together with a top-down error signal determine parameter updates. Our experiments show that incorporating delays improves accuracy by up to 20% over a weights-only baseline, and for networks with similar parameter counts, jointly learning weights and delays yields up to 14% higher accuracy. On the SHD speech recognition dataset, our method achieves similar accuracy to offline backpropagation-based approaches. Compared to state-of-the-art methods, it reduces model size by 6.6x and inference latency by 67%, with only a 2.4% drop in classification accuracy. Our findings benefit the design of power and area-constrained neuromorphic processors by enabling on-device learning and lowering memory requirements.

Related papers

• An Exact Gradient Framework for Training Spiking Neural Networks [0.7366405857677227]
  Spiking neural networks inherently rely on the precise timing of discrete spike events for information processing.<n>We propose an event-driven learning framework that computes exact loss gradients with respect to synaptic weights and transmission delays.<n>Experiments on multiple benchmarks demonstrate significant gains in accuracy (up to 7%), timing precision, and robustness compared to existing methods.
  arXiv  Detail & Related papers  (2025-07-08T11:55:27Z)
• A Stable Whitening Optimizer for Efficient Neural Network Training [99.7641280234926]
  Building on the Shampoo family of algorithms, we identify and alleviate three key issues, resulting in the proposed SPlus method.<n>First, we find that naive Shampoo is prone to divergence when matrix-inverses are cached for long periods.<n>Second, we adapt a shape-aware scaling to enable learning rate transfer across network width.<n>Third, we find that high learning rates result in large parameter noise, and propose a simple iterate-averaging scheme which unblocks faster learning.
  arXiv  Detail & Related papers  (2025-06-08T18:43:31Z)
• Efficient Event-based Delay Learning in Spiking Neural Networks [0.1350479308585481]
  Spiking Neural Networks (SNNs) compute using sparse communication and are attracting increased attention.<n>We propose a novel event-based training method for SNNs with delays, grounded in the EventProp formalism.<n>Our method supports multiple spikes per neuron and, to the best of our knowledge, is the first delay learning algorithm to be applied to recurrent SNNs.
  arXiv  Detail & Related papers  (2025-01-13T13:44:34Z)
• Learning Delays Through Gradients and Structure: Emergence of Spatiotemporal Patterns in Spiking Neural Networks [0.06752396542927405]
  We present a Spiking Neural Network (SNN) model that incorporates learnable synaptic delays through two approaches. In the latter approach, the network selects and prunes connections, optimizing the delays in sparse connectivity settings. Our results demonstrate the potential of combining delay learning with dynamic pruning to develop efficient SNN models for temporal data processing.
  arXiv  Detail & Related papers  (2024-07-07T11:55:48Z)
• DelGrad: Exact event-based gradients for training delays and weights on spiking neuromorphic hardware [1.5226147562426895]
  Spiking neural networks (SNNs) inherently rely on the timing of signals for representing and processing information.<n>We propose DelGrad, an event-based method to compute exact loss gradients for both synaptic weights and delays.<n>We experimentally demonstrate the memory efficiency and accuracy benefits of adding delays to SNNs on noisy mixed-signal hardware.
  arXiv  Detail & Related papers  (2024-04-30T00:02:34Z)
• Learning to Learn with Generative Models of Neural Network Checkpoints [71.06722933442956]
  We construct a dataset of neural network checkpoints and train a generative model on the parameters. We find that our approach successfully generates parameters for a wide range of loss prompts. We apply our method to different neural network architectures and tasks in supervised and reinforcement learning.
  arXiv  Detail & Related papers  (2022-09-26T17:59:58Z)
• Can pruning improve certified robustness of neural networks? [106.03070538582222]
  We show that neural network pruning can improve empirical robustness of deep neural networks (NNs) Our experiments show that by appropriately pruning an NN, its certified accuracy can be boosted up to 8.2% under standard training. We additionally observe the existence of certified lottery tickets that can match both standard and certified robust accuracies of the original dense models.
  arXiv  Detail & Related papers  (2022-06-15T05:48:51Z)
• Axonal Delay As a Short-Term Memory for Feed Forward Deep Spiking Neural Networks [3.985532502580783]
  Recent studies have found that the time delay of neurons plays an important role in the learning process. configuring the precise timing of the spike is a promising direction for understanding and improving the transmission process of temporal information in SNNs. In this paper, we verify the effectiveness of integrating time delay into supervised learning and propose a module that modulates the axonal delay through short-term memory.
  arXiv  Detail & Related papers  (2022-04-20T16:56:42Z)
• SignalNet: A Low Resolution Sinusoid Decomposition and Estimation Network [79.04274563889548]
  We propose SignalNet, a neural network architecture that detects the number of sinusoids and estimates their parameters from quantized in-phase and quadrature samples. We introduce a worst-case learning threshold for comparing the results of our network relative to the underlying data distributions. In simulation, we find that our algorithm is always able to surpass the threshold for three-bit data but often cannot exceed the threshold for one-bit data.
  arXiv  Detail & Related papers  (2021-06-10T04:21:20Z)
• Neural Architecture Search For LF-MMI Trained Time Delay Neural Networks [61.76338096980383]
  A range of neural architecture search (NAS) techniques are used to automatically learn two types of hyper- parameters of state-of-the-art factored time delay neural networks (TDNNs) These include the DARTS method integrating architecture selection with lattice-free MMI (LF-MMI) TDNN training. Experiments conducted on a 300-hour Switchboard corpus suggest the auto-configured systems consistently outperform the baseline LF-MMI TDNN systems.
  arXiv  Detail & Related papers  (2020-07-17T08:32:11Z)
• Rectified Linear Postsynaptic Potential Function for Backpropagation in Deep Spiking Neural Networks [55.0627904986664]
  Spiking Neural Networks (SNNs) usetemporal spike patterns to represent and transmit information, which is not only biologically realistic but also suitable for ultra-low-power event-driven neuromorphic implementation. This paper investigates the contribution of spike timing dynamics to information encoding, synaptic plasticity and decision making, providing a new perspective to design of future DeepSNNs and neuromorphic hardware systems.
  arXiv  Detail & Related papers  (2020-03-26T11:13:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.