S3T-Former: A Purely Spike-Driven State-Space Topology Transformer for Skeleton Action Recognition

• URL: http://arxiv.org/abs/2603.18062v2
• Date: Fri, 20 Mar 2026 01:48:09 GMT
• Title: S3T-Former: A Purely Spike-Driven State-Space Topology Transformer for Skeleton Action Recognition
• Abstract summary: Spiking State-Space Topology Transformer (S3T-Former) is first purely spike-driven Transformer architecture specifically designed for energy-efficient skeleton action recognition.<n>To achieve true topological and temporal sparsity, we introduce Lateral Spiking Topology Routing (LSTR) for on-demand conditional spike propagation.<n>Experiments on multiple large-scale datasets demonstrate that S3T-Former achieves highly competitive accuracy while theoretically reducing energy consumption.
• Score: 11.456020141268832
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Skeleton-based action recognition is crucial for multimedia applications but heavily relies on power-hungry Artificial Neural Networks (ANNs), limiting their deployment on resource-constrained edge devices. Spiking Neural Networks (SNNs) provide an energy-efficient alternative; however, existing spiking models for skeleton data often compromise the intrinsic sparsity of SNNs by resorting to dense matrix aggregations, heavy multimodal fusion modules, or non-sparse frequency domain transformations. Furthermore, they severely suffer from the short-term amnesia of spiking neurons. In this paper, we propose the Spiking State-Space Topology Transformer (S3T-Former), which, to the best of our knowledge, is the first purely spike-driven Transformer architecture specifically designed for energy-efficient skeleton action recognition. Rather than relying on heavy fusion overhead, we formulate a Multi-Stream Anatomical Spiking Embedding (M-ASE) that acts as a generalized kinematic differential operator, elegantly transforming multimodal skeleton features into heterogeneous, highly sparse event streams. To achieve true topological and temporal sparsity, we introduce Lateral Spiking Topology Routing (LSTR) for on-demand conditional spike propagation, and a Spiking State-Space (S3) Engine to systematically capture long-range temporal dynamics without non-sparse spectral workarounds. Extensive experiments on multiple large-scale datasets demonstrate that S3T-Former achieves highly competitive accuracy while theoretically reducing energy consumption compared to classic ANNs, establishing a new state-of-the-art for energy-efficient neuromorphic action recognition.

Related papers

• Elastic Spiking Transformers for Efficient Gesture Understanding [6.52225846892392]
  Spiking Neural Networks (SNNs) offer energy-efficient processing of event-based sensor data for healthcare applications.<n>We introduce the Elastic Spiking Transformer, a runtime-adaptive architecture that brings elasticity into the spiking paradigm.<n>We evaluate the approach on CIFAR10/100, CIFAR10-DVS, and the EHWGesture clinical gesture understanding dataset.
  arXiv  Detail & Related papers  (2026-05-04T12:35:52Z)
• Brain-Inspired Multimodal Spiking Neural Network for Image-Text Retrieval [79.04396446873932]
  Spiking neural networks (SNNs) have recently shown strong potential in unimodal visual and textual tasks.<n>Building a directly trained, low-energy, and high-performance SNN for multimodal applications such as image-text retrieval (ITR) remains highly challenging.<n>We present a brain-inspired Cross-Modal Spike Fusion network (CMSF) and apply it to ITR for the first time.
  arXiv  Detail & Related papers  (2026-03-25T08:41:07Z)
• Subtractive Modulative Network with Learnable Periodic Activations [59.89799070130572]
  We propose a novel, parameter-efficient Implicit Neural Representation architecture inspired by classical subtractive synthesis.<n>Our SMN achieves a PSNR of $40+$ dB on two image datasets, comparing favorably against state-of-the-art methods in terms of both reconstruction accuracy and parameter efficiency.
  arXiv  Detail & Related papers  (2026-02-18T10:20:50Z)
• Signal-SGN++: Topology-Enhanced Time-Frequency Spiking Graph Network for Skeleton-Based Action Recognition [0.23332469289621785]
  Spiking Neural Networks (SNNs) offer energy efficiency but remain limited in capturing temporal-frequency and topological dependencies of human motion.<n>This article proposes Signal-SGN++, a topology-aware spiking graph framework that integrates adaptivity with time-frequency spiking dynamics.
  arXiv  Detail & Related papers  (2025-12-22T09:16:04Z)
• PHASE-Net: Physics-Grounded Harmonic Attention System for Efficient Remote Photoplethysmography Measurement [63.007237197267834]
  Existing deep learning methods are mostly physiological monitoring and lack theoretical robustness.<n>We propose a physics-informed r paradigm derived from the Navier-Stokes equations of hemodynamics, showing that the pulse signal follows a second-order system.<n>This provides a theoretical justification for using a Temporal Conal Network (TCN)<n>Phase-Net achieves state-of-the-art performance with strong efficiency, offering a theoretically grounded and deployment-ready r solution.
  arXiv  Detail & Related papers  (2025-09-29T14:36:45Z)
• Fractional Spike Differential Equations Neural Network with Efficient Adjoint Parameters Training [63.3991315762955]
  Spiking Neural Networks (SNNs) draw inspiration from biological neurons to create realistic models for brain-like computation.<n>Most existing SNNs assume a single time constant for neuronal membrane voltage dynamics, modeled by first-order ordinary differential equations (ODEs) with Markovian characteristics.<n>We propose the Fractional SPIKE Differential Equation neural network (fspikeDE), which captures long-term dependencies in membrane voltage and spike trains through fractional-order dynamics.
  arXiv  Detail & Related papers  (2025-07-22T18:20:56Z)
• BrainSymphony: A Transformer-Driven Fusion of fMRI Time Series and Structural Connectivity [2.3486335708866606]
  BrainSymphony is a lightweight, parameter-efficient foundation model for neuroimaging.<n>It achieves state-of-the-art performance while being pre-trained on significantly smaller public datasets.<n>BrainSymphony establishes that architecturally-aware, multimodal models can surpass their larger counterparts.
  arXiv  Detail & Related papers  (2025-06-23T06:00:21Z)
• Signal-SGN: A Spiking Graph Convolutional Network for Skeletal Action Recognition via Learning Temporal-Frequency Dynamics [0.7326395356941241]
  Spiking Neural Networks (SNNs) struggle to model skeleton dynamics, leading to suboptimal solutions.<n>We propose Signal-SGN (Spiking Graph Convolutional Network), which utilizes the temporal dimension of skeleton sequences as the spike time steps.<n> Experiments across three large-scale datasets reveal Signal-SGN exceeding state-of-the-art SNN-based methods in accuracy and computational efficiency.
  arXiv  Detail & Related papers  (2024-08-03T07:47:16Z)
• Contextualizing MLP-Mixers Spatiotemporally for Urban Data Forecast at Scale [54.15522908057831]
  We propose an adapted version of the computationally-Mixer for STTD forecast at scale. Our results surprisingly show that this simple-yeteffective solution can rival SOTA baselines when tested on several traffic benchmarks. Our findings contribute to the exploration of simple-yet-effective models for real-world STTD forecasting.
  arXiv  Detail & Related papers  (2023-07-04T05:19:19Z)
• Multi-Scale Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition [140.18376685167857]
  A simple yet effective multi-scale semantics-guided neural network is proposed for skeleton-based action recognition. MS-SGN achieves the state-of-the-art performance on the NTU60, NTU120, and SYSU datasets.
  arXiv  Detail & Related papers  (2021-11-07T03:50:50Z)
• Perceptron Synthesis Network: Rethinking the Action Scale Variances in Videos [48.57686258913474]
  Video action recognition has been partially addressed by the CNNs stacking of fixed-size 3D kernels. We propose to learn the optimal-scale kernels from the data. An textitaction perceptron synthesizer is proposed to generate the kernels from a bag of fixed-size kernels.
  arXiv  Detail & Related papers  (2020-07-22T14:22:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.