Fire on Motion: Optimizing Video Pass-bands for Efficient Spiking Action Recognition

• URL: http://arxiv.org/abs/2601.22675v1
• Date: Fri, 30 Jan 2026 07:47:23 GMT
• Title: Fire on Motion: Optimizing Video Pass-bands for Efficient Spiking Action Recognition
• Abstract summary: Spiking neural networks (SNNs) have gained traction in vision due to their energy efficiency, bio-plausibility, and inherent temporal processing.<n>However, SNNs still underperform on dynamic video tasks compared to artificial neural networks (ANNs)
• Score: 21.25634534419545
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Spiking neural networks (SNNs) have gained traction in vision due to their energy efficiency, bio-plausibility, and inherent temporal processing. Yet, despite this temporal capacity, most progress concentrates on static image benchmarks, and SNNs still underperform on dynamic video tasks compared to artificial neural networks (ANNs). In this work, we diagnose a fundamental pass-band mismatch: Standard spiking dynamics behave as a temporal low pass that emphasizes static content while attenuating motion bearing bands, where task relevant information concentrates in dynamic tasks. This phenomenon explains why SNNs can approach ANNs on static tasks yet fall behind on tasks that demand richer temporal understanding.To remedy this, we propose the Pass-Bands Optimizer (PBO), a plug-and-play module that optimizes the temporal pass-band toward task-relevant motion bands. PBO introduces only two learnable parameters, and a lightweight consistency constraint that preserves semantics and boundaries, incurring negligible computational overhead and requires no architectural changes. PBO deliberately suppresses static components that contribute little to discrimination, effectively high passing the stream so that spiking activity concentrates on motion bearing content. On UCF101, PBO yields over ten percentage points improvement. On more complex multi-modal action recognition and weakly supervised video anomaly detection, PBO delivers consistent and significant gains, offering a new perspective for SNN based video processing and understanding.

Related papers

• SpikeTAD: Spiking Neural Networks for End-to-End Temporal Action Detection [24.33424613315401]
  Spiking Neural Networks (SNNs) have shown bioplausibility and low power advantages over Artificial Neural Networks (ANNs)<n>We propose the first SNN-based end-to-end temporal action detection architecture coined as SpikeTAD.<n>SpikeTAD achieves an average mAP of 67.2% in THUMOS14 and 37.42% in ActivityNet-1.3, demonstrating the feasibility of a low-power TAD model.
  arXiv  Detail & Related papers  (2026-06-10T12:57:05Z)
• Neural Dynamics Self-Attention for Spiking Transformers [59.57278823240336]
  Spiking Neural Networks (SNNs) with Transformer architectures offer a promising pathway to balance energy efficiency and performance.<n>Existing Spiking Transformers face two critical challenges: (i) a substantial performance gap compared to their Artificial Neural Networks (ANNs) counterparts and (ii) high memory overhead during inference.<n>We propose LRF-Dyn, which uses spiking neurons with localized receptive fields to compute attention while reducing memory requirements.
  arXiv  Detail & Related papers  (2026-03-09T12:23:37Z)
• PTS-SNN: A Prompt-Tuned Temporal Shift Spiking Neural Networks for Efficient Speech Emotion Recognition [12.087823767638788]
  Speech Emotion Recognition (SER) is widely deployed in Human-Computer Interaction, yet the high computational cost hinders their implementation on resource-constrained edge devices.<n>We propose Prompt-Tuned Spiking Neural Networks (PTS-SNN), a parameter-efficient neuromorphic adaptation with spiking dynamics.
  arXiv  Detail & Related papers  (2026-02-09T03:29:16Z)
• Unleashing Temporal Capacity of Spiking Neural Networks through Spatiotemporal Separation [67.69345363409835]
  Spiking Neural Networks (SNNs) are considered naturally suited for temporal processing, with membrane potential propagation widely regarded as the core temporal modeling mechanism.<n>We design Non-Stateful (NS) models progressively removing membrane propagation to its stage-wise role. Experiments reveal a counterintuitive phenomenon: moderate removal in shallow layers improves performance, while excessive removal causes collapse.
  arXiv  Detail & Related papers  (2025-12-05T07:05:53Z)
• Ultra-Low-Latency Spiking Neural Networks with Temporal-Dependent Integrate-and-Fire Neuron Model for Objects Detection [9.928561993466458]
  Spiking Neural Networks (SNNs) are characterized by minimal power consumption and swift inference capabilities.<n>We propose a delay-spike approach to mitigate the issue of residual membrane potential caused by heterogeneous spiking patterns.<n>We demonstrate that our method achieves more precise feature representation with lower time-steps, enabling high performance and ultra-low latency in visual detection tasks.
  arXiv  Detail & Related papers  (2025-08-28T03:32:45Z)
• Learning with Spike Synchrony in Spiking Neural Networks [3.8506283985103447]
  Spiking neural networks (SNNs) promise energy-efficient computation by mimicking biological neural dynamics.<n>We introduce spike-synchrony-dependent plasticity (SSDP), a training approach that adjusts synaptic weights based on the degree of neural firing rather than spike timing.
  arXiv  Detail & Related papers  (2025-04-14T04:01:40Z)
• Spiking Meets Attention: Efficient Remote Sensing Image Super-Resolution with Attention Spiking Neural Networks [86.28783985254431]
  Spiking neural networks (SNNs) are emerging as a promising alternative to traditional artificial neural networks (ANNs)<n>We propose SpikeSR, which achieves state-of-the-art performance across various remote sensing benchmarks such as AID, DOTA, and DIOR.
  arXiv  Detail & Related papers  (2025-03-06T09:06:06Z)
• Rethinking Spiking Neural Networks from an Ensemble Learning Perspective [4.823440259626247]
  Spiking neural networks (SNNs) exhibit superior energy efficiency but suffer from limited performance.<n>In this paper, we consider SNNs as ensembles of temporalworks that share architectures and weights.<n>We promote the consistency of the initial membrane potential distribution and output through membrane potential smoothing and temporally adjacent subnetwork guidance.
  arXiv  Detail & Related papers  (2025-02-20T03:15:52Z)
• Activation-wise Propagation: A Universal Strategy to Break Timestep Constraints in Spiking Neural Networks for 3D Data Processing [29.279985043923386]
  We introduce Activation-wise Membrane Potential Propagation (AMP2), a novel state update mechanism for spiking neurons.<n>Inspired by skip connections in deep networks, AMP2 incorporates the membrane potential of neurons into network, eliminating the need for iterative updates.<n>Our method achieves significant improvements across various 3D modalities, including 3D point clouds and event streams.
  arXiv  Detail & Related papers  (2025-02-18T11:52:25Z)
• Towards Low-latency Event-based Visual Recognition with Hybrid Step-wise Distillation Spiking Neural Networks [50.32980443749865]
  Spiking neural networks (SNNs) have garnered significant attention for their low power consumption and high biologicalability. Current SNNs struggle to balance accuracy and latency in neuromorphic datasets. We propose Step-wise Distillation (HSD) method, tailored for neuromorphic datasets.
  arXiv  Detail & Related papers  (2024-09-19T06:52:34Z)
• SpikeMba: Multi-Modal Spiking Saliency Mamba for Temporal Video Grounding [50.337896542603524]
  We introduce SpikeMba: a multi-modal spiking saliency mamba for temporal video grounding. Our approach integrates Spiking Neural Networks (SNNs) with state space models (SSMs) to leverage their unique advantages. Our experiments demonstrate the effectiveness of SpikeMba, which consistently outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-04-01T15:26:44Z)
• Ultra-low Latency Spiking Neural Networks with Spatio-Temporal Compression and Synaptic Convolutional Block [4.081968050250324]
  Spiking neural networks (SNNs) have neuro-temporal information capability, low processing feature, and high biological plausibility. Neuro-MNIST, CIFAR10-S, DVS128 gesture datasets need to aggregate individual events into frames with a higher temporal resolution for event stream classification. We propose a processing-temporal compression method to aggregate individual events into a few time steps of NIST current to reduce the training and inference latency.
  arXiv  Detail & Related papers  (2022-03-18T15:14:13Z)
• A Prospective Study on Sequence-Driven Temporal Sampling and Ego-Motion Compensation for Action Recognition in the EPIC-Kitchens Dataset [68.8204255655161]
  Action recognition is one of the top-challenging research fields in computer vision. ego-motion recorded sequences have become of important relevance. The proposed method aims to cope with it by estimating this ego-motion or camera motion.
  arXiv  Detail & Related papers  (2020-08-26T14:44:45Z)
• Learn to cycle: Time-consistent feature discovery for action recognition [83.43682368129072]
  Generalizing over temporal variations is a prerequisite for effective action recognition in videos. We introduce Squeeze Re Temporal Gates (SRTG), an approach that favors temporal activations with potential variations. We show consistent improvement when using SRTPG blocks, with only a minimal increase in the number of GFLOs.
  arXiv  Detail & Related papers  (2020-06-15T09:36:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.