PSTTS: A Plug-and-Play Token Selector for Efficient Event-based Spatio-temporal Representation Learning

• URL: http://arxiv.org/abs/2509.22481v1
• Date: Fri, 26 Sep 2025 15:30:00 GMT
• Title: PSTTS: A Plug-and-Play Token Selector for Efficient Event-based Spatio-temporal Representation Learning
• Abstract summary: We propose Progressive Spatio-temporal Token Selection (PSTTS) for event data.<n>PSTTS exploits the temporal-temporal distribution characteristics embedded in raw event data to effectively identify and discard redundant tokens.<n>PSTTS reduces FLOPs by 29-43.6% and increases FPS by 21.6-41.3% the DailyDVS-200 dataset.
• Score: 25.271901669843363
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Mainstream event-based spatio-temporal representation learning methods typically process event streams by converting them into sequences of event frames, achieving remarkable performance. However, they neglect the high spatial sparsity and inter-frame motion redundancy inherent in event frame sequences, leading to significant computational overhead. Existing token sparsification methods for RGB videos rely on unreliable intermediate token representations and neglect the influence of event noise, making them ineffective for direct application to event data. In this paper, we propose Progressive Spatio-Temporal Token Selection (PSTTS), a Plug-and-Play module for event data without introducing any additional parameters. PSTTS exploits the spatio-temporal distribution characteristics embedded in raw event data to effectively identify and discard spatio-temporal redundant tokens, achieving an optimal trade-off between accuracy and efficiency. Specifically, PSTTS consists of two stages, Spatial Token Purification and Temporal Token Selection. Spatial Token Purification discards noise and non-event regions by assessing the spatio-temporal consistency of events within each event frame to prevent interference with subsequent temporal redundancy evaluation. Temporal Token Selection evaluates the motion pattern similarity between adjacent event frames, precisely identifying and removing redundant temporal information. We apply PSTTS to four representative backbones UniformerV2, VideoSwin, EVMamba, and ExACT on the HARDVS, DailyDVS-200, and SeACT datasets. Experimental results demonstrate that PSTTS achieves significant efficiency improvements. Specifically, PSTTS reduces FLOPs by 29-43.6% and increases FPS by 21.6-41.3% on the DailyDVS-200 dataset, while maintaining task accuracy. Our code will be available.

Related papers

• Following the Flow: Advection-Consistent Modeling for Event-based Small Object Detection [2.925778409623925]
  Event cameras enable high-frequency visual perception with microsecond latency, offering advantages for dynamic scenes.<n>However, event-based small object detection remains challenging due to sparse asynchronous measurements and weak object responses that are easily disrupted by noise.<n>We propose a physics-guided advection-consistent framework, termed PACT, which formulates event evolution as a motion-driven feature transport process.
  arXiv  Detail & Related papers  (2026-06-21T07:56:34Z)
• E.M.Ground: A Temporal Grounding Vid-LLM with Holistic Event Perception and Matching [87.38371267983263]
  Temporal Video Grounding aims to precisely localize time segments corresponding to query events.<n>E.M.Ground is a novel Vid-LLM for TVG that focuses on holistic and coherent event perception.<n>E.M.Ground consistently outperforms state-of-the-art Vid-LLMs by significant margins.
  arXiv  Detail & Related papers  (2026-02-05T02:16:00Z)
• PPMStereo: Pick-and-Play Memory Construction for Consistent Dynamic Stereo Matching [51.98089287914147]
  textbfPick-and-textbflay textbfMemory (PM) construction module for dynamic bfStereo matching, dubbed as bftextPPMStereo.<n>Inspired by the two-stage decision-making process in humans, we propose a textbfPick-and-textbflay textbfMemory (PM) construction module for dynamic bfStereo matching, dubbed as bftextPPMStereo.
  arXiv  Detail & Related papers  (2025-10-23T03:52:39Z)
• Focus Through Motion: RGB-Event Collaborative Token Sparsification for Efficient Object Detection [56.88160531995454]
  Existing RGB-Event detection methods process the low-information regions of both modalities uniformly during feature extraction and fusion.<n>We propose FocusMamba, which performs adaptive collaborative sparsification of multimodal features.<n>Experiments on the DSEC-Det and PKU-DAVIS-SOD datasets demonstrate that the proposed method achieves superior performance in both accuracy and efficiency.
  arXiv  Detail & Related papers  (2025-09-04T04:18:46Z)
• Efficient Event-Based Semantic Segmentation via Exploiting Frame-Event Fusion: A Hybrid Neural Network Approach [13.681130802391223]
  Event cameras have recently been introduced into image semantic segmentation, owing to their high temporal resolution and other advantageous properties.<n>We propose an efficient hybrid framework for image semantic segmentation, comprising a Spiking Network branch for events and an Artificial Neural Network branch for frames.<n>Specifically, we introduce three specialized modules to facilitate the interaction between these two branches: the Adaptive Temporal Weighting (ATW), the Event-Driven Sparse (EDS), and the Channel Selection Fusion (CSF) module.
  arXiv  Detail & Related papers  (2025-07-04T18:30:28Z)
• Self-Supervised Event Representations: Towards Accurate, Real-Time Perception on SoC FPGAs [0.0]
  Event cameras offer significant advantages over traditional frame-based sensors.<n>The effective processing of their sparse, asynchronous event streams remains challenging.<n>This paper introduces a novel Self-Supervised Event Representation (SSER) method.
  arXiv  Detail & Related papers  (2025-05-12T13:32:08Z)
• Labits: Layered Bidirectional Time Surfaces Representation for Event Camera-based Continuous Dense Trajectory Estimation [1.3416369506987165]
  Event cameras capture dynamic scenes with high temporal resolution and low latency.<n>We introduce Labits: Layered Bidirectional Time Surfaces, a simple yet elegant representation designed to retain all these features.<n>Our approach achieves an impressive 49% reduction in trajectory end-point error (TEPE) compared to the previous state-of-the-art on the MultiFlow dataset.
  arXiv  Detail & Related papers  (2024-12-12T01:11:50Z)
• EventCrab: Harnessing Frame and Point Synergy for Event-based Action Recognition and Beyond [61.10181853363728]
  Event-based Action Recognition (EAR) possesses the advantages of high-temporal and privacy preservation compared with traditional action recognition.<n>We present EventCrab, a framework that adeptly integrates the "lighter" frame-specific networks for dense event frames with the "heavier" point-specific networks for sparse event points.<n>Experiments on four datasets demonstrate the significant performance of our proposed EventCrab.
  arXiv  Detail & Related papers  (2024-11-27T13:28:57Z)
• PASS: Path-selective State Space Model for Event-based Recognition [12.651829415097758]
  Event cameras are bio-inspired sensors with advantages, such as high temporal resolution.<n>We present our PASS framework, exhibiting superior capacity for high event modeling.<n>Our key insight is to learn adaptively encoded event features via the state space models.
  arXiv  Detail & Related papers  (2024-09-25T14:08:37Z)
• Implicit Event-RGBD Neural SLAM [54.74363487009845]
  Implicit neural SLAM has achieved remarkable progress recently. Existing methods face significant challenges in non-ideal scenarios. We propose EN-SLAM, the first event-RGBD implicit neural SLAM framework.
  arXiv  Detail & Related papers  (2023-11-18T08:48:58Z)
• Dual Memory Aggregation Network for Event-Based Object Detection with Learnable Representation [79.02808071245634]
  Event-based cameras are bio-inspired sensors that capture brightness change of every pixel in an asynchronous manner. Event streams are divided into grids in the x-y-t coordinates for both positive and negative polarity, producing a set of pillars as 3D tensor representation. Long memory is encoded in the hidden state of adaptive convLSTMs while short memory is modeled by computing spatial-temporal correlation between event pillars.
  arXiv  Detail & Related papers  (2023-03-17T12:12:41Z)
• Ev-TTA: Test-Time Adaptation for Event-Based Object Recognition [7.814941658661939]
  Ev-TTA is a simple, effective test-time adaptation for event-based object recognition. Our formulation can be successfully applied regardless of input representations and extended into regression tasks.
  arXiv  Detail & Related papers  (2022-03-23T07:43:44Z)
• Ultra-low Latency Spiking Neural Networks with Spatio-Temporal Compression and Synaptic Convolutional Block [4.081968050250324]
  Spiking neural networks (SNNs) have neuro-temporal information capability, low processing feature, and high biological plausibility. Neuro-MNIST, CIFAR10-S, DVS128 gesture datasets need to aggregate individual events into frames with a higher temporal resolution for event stream classification. We propose a processing-temporal compression method to aggregate individual events into a few time steps of NIST current to reduce the training and inference latency.
  arXiv  Detail & Related papers  (2022-03-18T15:14:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.