Dynamic Pondering Sparsity-aware Mixture-of-Experts Transformer for Event Stream based Visual Object Tracking

• URL: http://arxiv.org/abs/2605.06112v1
• Date: Thu, 07 May 2026 12:25:03 GMT
• Title: Dynamic Pondering Sparsity-aware Mixture-of-Experts Transformer for Event Stream based Visual Object Tracking
• Abstract summary: RGB-based trackers are vulnerable to challenging imaging conditions, such as low illumination and fast motion.<n>Event cameras offer a promising alternative by capturing pixel-wise brightness changes, providing high dynamic range and high temporal resolution.<n>We propose an event sparsity-aware tracking framework that explicitly models event-density variations across multiple temporal scales.
• Score: 44.11576998237289
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite significant progress, RGB-based trackers remain vulnerable to challenging imaging conditions, such as low illumination and fast motion. Event cameras offer a promising alternative by asynchronously capturing pixel-wise brightness changes, providing high dynamic range and high temporal resolution. However, existing event-based trackers often neglect the intrinsic spatial sparsity and temporal density of event data, while relying on a single fixed temporal-window sampling strategy that is suboptimal under varying motion dynamics. In this paper, we propose an event sparsity-aware tracking framework that explicitly models event-density variations across multiple temporal scales. Specifically, the proposed framework progressively injects sparse, medium-density, and dense event search regions into a three-stage Vision Transformer backbone, enabling hierarchical multi-density feature learning. Furthermore, we introduce a sparsity-aware Mixture-of-Experts module to encourage expert specialization under different sparsity patterns, and design a dynamic pondering strategy to adaptively adjust the inference depth according to tracking difficulty. Extensive experiments on FE240hz, COESOT, and EventVOT demonstrate that the proposed approach achieves a favorable trade-off between tracking accuracy and computational efficiency. The source code will be released on https://github.com/Event-AHU/OpenEvTracking.

Related papers

• GoStop: Reinforcement Learning for Adaptive Temporal Aggregation in Event-Based Feature Tracking [71.9852231453445]
  Event cameras enable low-latency and motion-robust perception, making them well-suited for feature tracking under fast and non-linear motion.<n>Existing event-based feature tracking methods rely on fixed rules based on hand-tuning for event accumulation.<n>We introduce reinforcement learning framework to adaptively control the accumulation process for online event-based feature tracking.
  arXiv  Detail & Related papers  (2026-07-17T07:18:07Z)
• Event Stream based Multi-Modal Video Anomaly Detection: A Benchmark Dataset and Algorithms [12.625916611891947]
  Video anomaly detection (VAD) is critical for automated surveillance but remains fragile under challenging conditions.<n>We propose EVAD, an event enhanced VAD framework that jointly exploits conventional video and event streams.<n>E VAD consistently outperforms methods, validating the effectiveness of event-based sensing for VAD in real world scenarios.
  arXiv  Detail & Related papers  (2026-07-10T05:58:52Z)
• EventGait: Towards Robust Gait Recognition with Event Streams [50.890621860023]
  Event cameras offer microsecond temporal resolution and high dynamic range, naturally capturing robust dynamic cues and suppressing static noise.<n>We propose textbfEventGait, an end-to-end dual-stream framework that separately models motion and shape while preserving the advantages of events.<n>Our approach sets a new state of the art on both synthesized and real-world event-based gait benchmarks, highlighting the robustness and potential of event-driven gait analysis.
  arXiv  Detail & Related papers  (2026-05-21T08:12:04Z)
• Towards Video Anomaly Detection from Event Streams: A Baseline and Benchmark Datasets [71.53287557600177]
  We take the first major step toward establishing event-based anomaly detection as a unified research direction.<n>We first construct multiple eventstream based benchmarks for video anomaly detection, featuring synchronized event and RGB recordings.<n>We then propose an EVent-centric Video Anomaly Detection framework, namely EWAD, with three key innovations.
  arXiv  Detail & Related papers  (2026-03-26T03:33:33Z)
• Decoupling Amplitude and Phase Attention in Frequency Domain for RGB-Event based Visual Object Tracking [51.31378940976401]
  Existing RGB-Event tracking approaches fail to fully exploit the unique advantages of event cameras.<n>We propose a novel tracking framework that performs early fusion in the frequency domain, enabling effective aggregation of high-frequency information from the event modality.<n>Experiments on three widely used RGB-Event tracking benchmark datasets, including FE108, FELT, and COESOT, demonstrate the high performance and efficiency of our method.
  arXiv  Detail & Related papers  (2026-01-03T01:10:17Z)
• Inference-Time Gaze Refinement for Micro-Expression Recognition: Enhancing Event-Based Eye Tracking with Motion-Aware Post-Processing [2.5465367830324905]
  Event-based eye tracking holds significant promise for fine-grained cognitive state inference.<n>We introduce a model-agnostic, inference-time refinement framework to enhance the output of existing event-based gaze estimation models.
  arXiv  Detail & Related papers  (2025-06-14T14:48:11Z)
• Towards Low-Latency Event Stream-based Visual Object Tracking: A Slow-Fast Approach [32.91982063297922]
  We propose a novel Slow-Fast Tracking paradigm that flexibly adapts to different operational requirements, termed SFTrack.<n>The proposed framework supports two complementary modes, i.e., a high-precision slow tracker for scenarios with sufficient computational resources, and an efficient fast tracker tailored for latency-aware, resource-constrained environments.<n>Our framework first performs graph-based representation learning from high-temporal-resolution event streams, and then integrates the learned graph-structured information into two FlashAttention-based vision backbones.
  arXiv  Detail & Related papers  (2025-05-19T09:37:23Z)
• SuperEIO: Self-Supervised Event Feature Learning for Event Inertial Odometry [6.552812892993662]
  Event cameras asynchronously output low-latency event streams, promising for state estimation in high-speed motion and challenging lighting conditions.<n>We propose SuperEIO, a novel framework that leverages the learning-based event-only detection and IMU measurements to achieve eventinertial odometry.<n>We evaluate our method extensively on multiple public datasets, demonstrating its superior accuracy and robustness compared to other state-of-the-art event-based methods.
  arXiv  Detail & Related papers  (2025-03-29T03:58:15Z)
• Dynamic Subframe Splitting and Spatio-Temporal Motion Entangled Sparse Attention for RGB-E Tracking [32.86991031493605]
  Event-based bionic camera captures dynamic scenes with high temporal resolution and high dynamic range. We propose a dynamic event subframe splitting strategy to split the event stream into more fine-grained event clusters. Based on this, we design an event-based sparse attention mechanism to enhance the interaction of event features in temporal and spatial dimensions.
  arXiv  Detail & Related papers  (2024-09-26T06:12:08Z)
• An Event-Oriented Diffusion-Refinement Method for Sparse Events Completion [36.64856578682197]
  Event cameras or dynamic vision sensors (DVS) record asynchronous response to brightness changes instead of conventional intensity frames. We propose an inventive event completion sequence approach conforming to unique characteristics of event data in both the processing stage and the output form. Specifically, we treat event streams as 3D event clouds in thetemporal domain, develop a diffusion-based generative model to generate dense clouds in a coarse-to-fine manner, and recover exact timestamps to maintain the temporal resolution of raw data successfully.
  arXiv  Detail & Related papers  (2024-01-06T08:09:54Z)
• Implicit Event-RGBD Neural SLAM [54.74363487009845]
  Implicit neural SLAM has achieved remarkable progress recently. Existing methods face significant challenges in non-ideal scenarios. We propose EN-SLAM, the first event-RGBD implicit neural SLAM framework.
  arXiv  Detail & Related papers  (2023-11-18T08:48:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.