SHaRe-SSM: An Oscillatory Spiking Neural Network for Target Variable Modeling in Long Sequences

• URL: http://arxiv.org/abs/2510.14386v1
• Date: Thu, 16 Oct 2025 07:37:59 GMT
• Title: SHaRe-SSM: An Oscillatory Spiking Neural Network for Target Variable Modeling in Long Sequences
• Authors: Kartikay Agrawal, Abhijeet Vikram, Vedant Sharma, Vaishnavi N., Ayon Borthakur,
• Abstract summary: State space models (SSMs) have evolved as a powerful alternative to transformers for target modeling in long sequences.<n>We design SHaRe-SSM for target variable modeling (including both classification and regression) for very-long-range sequences.<n>Our network shows superior performance on even a 50k sequence while being significantly energy-efficient.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In recent years, with the emergence of large models, there has been a significant interest in spiking neural networks (SNNs) primarily due to their energy efficiency, multiplication-free, and sparse event-based deep learning. Similarly, state space models (SSMs) in varying designs have evolved as a powerful alternative to transformers for target modeling in long sequences, thereby overcoming the quadratic dependence on sequence length of a transformer. Inspired by this progress, we here design SHaRe-SSM (Spiking Harmonic Resonate and Fire State Space Model), for target variable modeling (including both classification and regression) for very-long-range sequences. Our second-order spiking SSM, on average, performs better than transformers or first-order SSMs while circumventing multiplication operations, making it ideal for resource-constrained applications. The proposed block consumes $73 \times$ less energy than second-order ANN-based SSMs for an 18k sequence, while retaining performance. To ensure learnability over the long-range sequences, we propose exploiting the stable and efficient implementation of the dynamical system using parallel scans. Moreover, for the first time, we propose a kernel-based spiking regressor using resonate and fire neurons for very long-range sequences. Our network shows superior performance on even a 50k sequence while being significantly energy-efficient. In addition, we conducted a systematic analysis of the impact of heterogeneity, dissipation, and conservation in resonate-and-fire SSMs.

Related papers

• MS-SSM: A Multi-Scale State Space Model for Efficient Sequence Modeling [60.648359990090846]
  State-space models (SSMs) have recently attention as an efficient alternative to computationally expensive attention-based models for sequence modeling.<n>This paper introduces a multi-scale SSM framework that represents sequence dynamics across multiple resolution and processing each resolution with specialized state-space dynamics.
  arXiv  Detail & Related papers  (2025-12-29T19:36:28Z)
• Systolic Array-based Accelerator for Structured State-Space Models [1.137896937254823]
  State-Space Models (SSMs) process very long data sequences more efficiently than recurrent and Transformer-based models.<n>In this paper, we introduce a specialized hardware accelerator, EpochCore, for accelerating SSMs.<n>EpochCore achieves on average 2000x improvement in performance on LRA datasets compared to a GPU.
  arXiv  Detail & Related papers  (2025-07-29T00:01:57Z)
• Sequential-Parallel Duality in Prefix Scannable Models [68.39855814099997]
  Recent developments have given rise to various models, such as Gated Linear Attention (GLA) and Mamba.<n>This raises a natural question: can we characterize the full class of neural sequence models that support near-constant-time parallel evaluation and linear-time, constant-space sequential inference?
  arXiv  Detail & Related papers  (2025-06-12T17:32:02Z)
• Structured State Space Model Dynamics and Parametrization for Spiking Neural Networks [0.8321953606016751]
  Multi-state spiking neurons offer compelling alternatives to conventional deep learning models.<n>State space models (SSMs) excel in long sequence processing using linear state-intrinsic recurrence resembling spiking neurons' subthreshold regime.<n>Here, we establish a mathematical bridge between SSMs and second-order spiking neuron models.
  arXiv  Detail & Related papers  (2025-06-04T13:54:02Z)
• Learning to Dissipate Energy in Oscillatory State-Space Models [51.98491034847041]
  State-space models (SSMs) are a class of networks for sequence learning.<n>We show that D-LinOSS consistently outperforms previous LinOSS methods on long-range learning tasks.
  arXiv  Detail & Related papers  (2025-05-17T23:15:17Z)
• Oscillatory State-Space Models [61.923849241099184]
  We propose Lineary State-Space models (LinOSS) for efficiently learning on long sequences.<n>A stable discretization, integrated over time using fast associative parallel scans, yields the proposed state-space model.<n>We show that LinOSS is universal, i.e., it can approximate any continuous and causal operator mapping between time-varying functions.
  arXiv  Detail & Related papers  (2024-10-04T22:00:13Z)
• PRF: Parallel Resonate and Fire Neuron for Long Sequence Learning in Spiking Neural Networks [6.545474731089018]
  We address the efficiency and performance challenges of long sequence learning in Spiking Neural Networks (SNNs) simultaneously. First, we propose a decoupled reset method for parallel spiking neuron training, reducing the typical Leaky Integrate-and-Fire (LIF) model's training time from $O(L2)$ to $O(Llog L)$. Secondly, to capture long-range dependencies, we propose a Parallel Resonate and Fire (PRF) neuron, which leverages an oscillating membrane potential driven by a resonate mechanism from a differentiable reset function in the complex domain
  arXiv  Detail & Related papers  (2024-10-04T15:51:56Z)
• SpikingSSMs: Learning Long Sequences with Sparse and Parallel Spiking State Space Models [19.04709216497077]
  We develop spiking state space models (SpikingSSMs) for long sequence learning.<n>Inspired by dendritic neuron structure, we hierarchically integrate neuronal dynamics with the original SSM block.<n>We propose a light-weight surrogate dynamic network which accurately predicts the after-reset membrane potential and compatible to learnable thresholds.
  arXiv  Detail & Related papers  (2024-08-27T09:35:49Z)
• LongVQ: Long Sequence Modeling with Vector Quantization on Structured Memory [63.41820940103348]
  Self-attention mechanism's computational cost limits its practicality for long sequences. We propose a new method called LongVQ to compress the global abstraction as a length-fixed codebook. LongVQ effectively maintains dynamic global and local patterns, which helps to complement the lack of long-range dependency issues.
  arXiv  Detail & Related papers  (2024-04-17T08:26:34Z)
• Learning Long Sequences in Spiking Neural Networks [0.0]
  Spiking neural networks (SNNs) take inspiration from the brain to enable energy-efficient computations. Recent interest in efficient alternatives to Transformers has given rise to state-of-the-art recurrent architectures named state space models (SSMs)
  arXiv  Detail & Related papers  (2023-12-14T13:30:27Z)
• Convolutional State Space Models for Long-Range Spatiotemporal Modeling [65.0993000439043]
  ConvS5 is an efficient variant for long-rangetemporal modeling. It significantly outperforms Transformers and ConvNISTTM on a long horizon Moving-Lab experiment while training 3X faster than ConvLSTM and generating samples 400X faster than Transformers.
  arXiv  Detail & Related papers  (2023-10-30T16:11:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.