SpikySpace: A Spiking State Space Model for Energy-Efficient Time Series Forecasting

• URL: http://arxiv.org/abs/2601.02411v1
• Date: Fri, 02 Jan 2026 13:10:53 GMT
• Title: SpikySpace: A Spiking State Space Model for Energy-Efficient Time Series Forecasting
• Abstract summary: SpikySpace is a spiking state-space model that reduces the quadratic cost in the attention block to linear time via selective scanning.<n>Because complex operations such as exponentials and divisions are costly on neuromorphic chips, we introduce simplified approximations of SiLU and Softplus.<n>In matched settings, SpikySpace reduces estimated energy consumption by 98.73% and 96.24% compared to two state-of-the-art transformer based approaches.
• Score: 9.976522013586244
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Time-series forecasting often operates under tight power and latency budgets in fields like traffic management, industrial condition monitoring, and on-device sensing. These applications frequently require near real-time responses and low energy consumption on edge devices. Spiking neural networks (SNNs) offer event-driven computation and ultra-low power by exploiting temporal sparsity and multiplication-free computation. Yet existing SNN-based time-series forecasters often inherit complex transformer blocks, thereby losing much of the efficiency benefit. To solve the problem, we propose SpikySpace, a spiking state-space model (SSM) that reduces the quadratic cost in the attention block to linear time via selective scanning. Further, we replace dense SSM updates with sparse spike trains and execute selective scans only on spike events, thereby avoiding dense multiplications while preserving the SSM's structured memory. Because complex operations such as exponentials and divisions are costly on neuromorphic chips, we introduce simplified approximations of SiLU and Softplus to enable a neuromorphic-friendly model architecture. In matched settings, SpikySpace reduces estimated energy consumption by 98.73% and 96.24% compared to two state-of-the-art transformer based approaches, namely iTransformer and iSpikformer, respectively. In standard time series forecasting datasets, SpikySpace delivers competitive accuracy while substantially reducing energy cost and memory traffic. As the first full spiking state-space model, SpikySpace bridges neuromorphic efficiency with modern sequence modeling, marking a practical and scalable path toward efficient time series forecasting systems.

Related papers

• Spike-driven Large Language Model [49.52947423982696]
  Spiking Neural Networks (SNNs) possess spike-driven characteristics.<n>Current Large Language Models (LLMs) are primarily based on large-scale dense matrix multiplications.<n>We propose SDLLM, a spike-driven large language model that eliminates dense matrix multiplications through sparse addition operations.
  arXiv  Detail & Related papers  (2026-04-11T17:58:35Z)
• Ge$^\ ext{2}$mS-T: Multi-Dimensional Grouping for Ultra-High Energy Efficiency in Spiking Transformer [84.8831358775386]
  Spiking Neural Networks (SNNs) offer superior energy efficiency over Artificial Neural Networks (ANNs)<n>Existing paradigms including ANN-SNN Conversion and Spatial-Temporal Backpropagation (STBP) suffer from inherent limitations.<n>We propose Ge$text2$mS-T, a novel architecture implementing grouped computation across temporal, spatial and network structure dimensions.
  arXiv  Detail & Related papers  (2026-04-10T02:58:46Z)
• Spatial PDE-aware Selective State-space with Nested Memory for Mobile Traffic Grid Forecasting [0.81996963503528]
  We propose NeST-S6, a convolutional selective state-space model (SSM) with a spatial PDE-aware core, implemented in a nested learning paradigm.<n>NeST-S6 speeds full-grid reconstruction by 32 times and reduces MAC resolutions by 4.3 times compared to competitive per-pixel scanning models.
  arXiv  Detail & Related papers  (2026-03-12T18:17:57Z)
• Neural Dynamics Self-Attention for Spiking Transformers [59.57278823240336]
  Spiking Neural Networks (SNNs) with Transformer architectures offer a promising pathway to balance energy efficiency and performance.<n>Existing Spiking Transformers face two critical challenges: (i) a substantial performance gap compared to their Artificial Neural Networks (ANNs) counterparts and (ii) high memory overhead during inference.<n>We propose LRF-Dyn, which uses spiking neurons with localized receptive fields to compute attention while reducing memory requirements.
  arXiv  Detail & Related papers  (2026-03-09T12:23:37Z)
• LiQSS: Post-Transformer Linear Quantum-Inspired State-Space Tensor Networks for Real-Time 6G [85.58816960936069]
  Proactive and agentic control in Sixth-Generation (6G) Open Radio Access Networks (O-RAN) requires control-grade prediction under stringent Near-Time (Near-RT) latency and computational constraints.<n>This paper investigates a post-Transformer paradigm for efficient radio telemetry forecasting.<n>We propose a quantum-inspired state-space tensor network that replaces self-attention with stable structured state-space dynamics kernels.
  arXiv  Detail & Related papers  (2026-01-18T12:08:38Z)
• SHaRe-SSM: An Oscillatory Spiking Neural Network for Target Variable Modeling in Long Sequences [0.0]
  State space models (SSMs) have evolved as a powerful alternative to transformers for target modeling in long sequences.<n>We design SHaRe-SSM for target variable modeling (including both classification and regression) for very-long-range sequences.<n>Our network shows superior performance on even a 50k sequence while being significantly energy-efficient.
  arXiv  Detail & Related papers  (2025-10-16T07:37:59Z)
• Systolic Array-based Accelerator for Structured State-Space Models [1.137896937254823]
  State-Space Models (SSMs) process very long data sequences more efficiently than recurrent and Transformer-based models.<n>In this paper, we introduce a specialized hardware accelerator, EpochCore, for accelerating SSMs.<n>EpochCore achieves on average 2000x improvement in performance on LRA datasets compared to a GPU.
  arXiv  Detail & Related papers  (2025-07-29T00:01:57Z)
• Decoupling Spatio-Temporal Prediction: When Lightweight Large Models Meet Adaptive Hypergraphs [12.867023510751787]
  STH-SepNet is a novel framework that decouples temporal and spatial expressiveness to both efficiency and precision.<n>S-SepNet offers a pragmatic and scalable solution for temporal prediction in real-world applications.<n>This work may provide a promising lightweight framework for temporal prediction, aiming to reduce computational demands and while enhancing predictive performance.
  arXiv  Detail & Related papers  (2025-05-26T07:37:39Z)
• STNMamba: Mamba-based Spatial-Temporal Normality Learning for Video Anomaly Detection [48.997518615379995]
  Video anomaly detection (VAD) has been extensively researched due to its potential for intelligent video systems.<n>Most existing methods based on CNNs and transformers still suffer from substantial computational burdens.<n>We propose a lightweight and effective Mamba-based network named STNMamba to enhance the learning of spatial-temporal normality.
  arXiv  Detail & Related papers  (2024-12-28T08:49:23Z)
• Dynamic Diffusion Transformer [67.13876021157887]
  Diffusion Transformer (DiT) has demonstrated superior performance but suffers from substantial computational costs. We propose Dynamic Diffusion Transformer (DyDiT), an architecture that dynamically adjusts its computation along both timestep and spatial dimensions during generation. With 3% additional fine-tuning, our method reduces the FLOPs of DiT-XL by 51%, accelerates generation by 1.73, and achieves a competitive FID score of 2.07 on ImageNet.
  arXiv  Detail & Related papers  (2024-10-04T14:14:28Z)
• STGformer: Efficient Spatiotemporal Graph Transformer for Traffic Forecasting [11.208740750755025]
  Traffic is a cornerstone of smart city management enabling efficient allocation and transportation planning. Deep learning, with its ability to capture complex nonlinear patterns in data, has emerged as a powerful tool for traffic forecasting. graph neural networks (GCNs) and transformer-based models have shown promise, but their computational demands often hinder their application to realworld networks. We propose a noveltemporal graph transformer (STG) architecture, enabling efficient modeling of both global and local traffic patterns while maintaining a manageable computational footprint.
  arXiv  Detail & Related papers  (2024-10-01T04:15:48Z)
• Spiking Transformer with Spatial-Temporal Attention [26.7175155847563]
  Spike-based Transformer presents a compelling and energy-efficient alternative to traditional Artificial Neural Network (ANN)-based Transformers.<n>We propose Spiking Transformer with Spatial-Temporal Attention (STAtten), a simple and straightforward architecture that efficiently integrates both spatial and temporal information in the self-attention mechanism.<n>Our method can be seamlessly integrated into existing spike-based transformers without architectural overhaul.
  arXiv  Detail & Related papers  (2024-09-29T20:29:39Z)
• Sparser is Faster and Less is More: Efficient Sparse Attention for Long-Range Transformers [58.5711048151424]
  We introduce SPARSEK Attention, a novel sparse attention mechanism designed to overcome computational and memory obstacles. Our approach integrates a scoring network and a differentiable top-k mask operator, SPARSEK, to select a constant number of KV pairs for each query. Experimental results reveal that SPARSEK Attention outperforms previous sparse attention methods.
  arXiv  Detail & Related papers  (2024-06-24T15:55:59Z)
• PDFormer: Propagation Delay-Aware Dynamic Long-Range Transformer for Traffic Flow Prediction [78.05103666987655]
  spatial-temporal Graph Neural Network (GNN) models have emerged as one of the most promising methods to solve this problem. We propose a novel propagation delay-aware dynamic long-range transFormer, namely PDFormer, for accurate traffic flow prediction. Our method can not only achieve state-of-the-art performance but also exhibit competitive computational efficiency.
  arXiv  Detail & Related papers  (2023-01-19T08:42:40Z)
• An advanced spatio-temporal convolutional recurrent neural network for storm surge predictions [73.4962254843935]
  We study the capability of artificial neural network models to emulate storm surge based on the storm track/size/intensity history. This study presents a neural network model that can predict storm surge, informed by a database of synthetic storm simulations.
  arXiv  Detail & Related papers  (2022-04-18T23:42:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.