Event-Adaptive State Transition and Gated Fusion for RGB-Event Object Tracking

• URL: http://arxiv.org/abs/2604.13426v1
• Date: Wed, 15 Apr 2026 02:51:35 GMT
• Title: Event-Adaptive State Transition and Gated Fusion for RGB-Event Object Tracking
• Abstract summary: MambaTrack is a multimodal and efficient tracking framework built upon a Dynamic State Space Model(DSSM)<n>A learnable scalar governs the state evolution rate, enabling differentiated modeling of sparse and dense event flows.<n>MambaTrack achieves state-of-the-art performance on the FE108 and FELT datasets.
• Score: 9.837735583393266
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing Vision Mamba-based RGB-Event(RGBE) tracking methods suffer from using static state transition matrices, which fail to adapt to variations in event sparsity. This rigidity leads to imbalanced modeling-underfitting sparse event streams and overfitting dense ones-thus degrading cross-modal fusion robustness. To address these limitations, we propose MambaTrack, a multimodal and efficient tracking framework built upon a Dynamic State Space Model(DSSM). Our contributions are twofold. First, we introduce an event-adaptive state transition mechanism that dynamically modulates the state transition matrix based on event stream density. A learnable scalar governs the state evolution rate, enabling differentiated modeling of sparse and dense event flows. Second, we develop a Gated Projection Fusion(GPF) module for robust cross-modal integration. This module projects RGB features into the event feature space and generates adaptive gates from event density and RGB confidence scores. These gates precisely control the fusion intensity, suppressing noise while preserving complementary information. Experiments show that MambaTrack achieves state-of-the-art performance on the FE108 and FELT datasets. Its lightweight design suggests potential for real-time embedded deployment.

Related papers

• Weaving Light and Time: Unified Harmonic-Geometric Representation Learning for Dense RGB-Event Parsing [55.13802890769086]
  We present Evita, the first unified backbone specifically engineered for dedicated dense RGB-Event parsing.<n>To achieve profound modal synergy, Evita embeds a suite of intrinsic co-learning modules directly into every encoder layer.<n>Evita establishes new state-of-the-art metrics while delivering a superior accuracy-latency trade-off for real-time multimodal perception.
  arXiv  Detail & Related papers  (2026-07-10T06:52:09Z)
• CMTFormer: Marrying Transformer with Hierarchical Information Interaction for RGB-Event Object Detection [17.487415888287]
  Event cameras capture sparse brightness changes with high temporal resolution and high dynamic range.<n>Previous multi-modal fusion techniques fail to handle the inherent heterogeneity between RGB frames and event streams.<n>We propose a Cross-Modal information inTeraction transFormer, which hierarchically integrates RGB and event information.
  arXiv  Detail & Related papers  (2026-06-28T00:49:05Z)
• Active Adversarial Perturbation-driven Associative Memory Retrieval for RGB-Event Visual Object Tracking [58.338848435042536]
  RGB-Event tracking improves localization by fusing RGB appearance textures and dense temporal motion cues from event sensors.<n>Real-world scenes suffer diverse structured signal degradations that hinder traditional multi-modal fusion.<n>We present a hierarchical perturbation and retrieval framework tailored for RGB-Event tracking with robustness against partial target missing and modal degradation.
  arXiv  Detail & Related papers  (2026-06-24T23:34:06Z)
• EventGait: Towards Robust Gait Recognition with Event Streams [50.890621860023]
  Event cameras offer microsecond temporal resolution and high dynamic range, naturally capturing robust dynamic cues and suppressing static noise.<n>We propose textbfEventGait, an end-to-end dual-stream framework that separately models motion and shape while preserving the advantages of events.<n>Our approach sets a new state of the art on both synthesized and real-world event-based gait benchmarks, highlighting the robustness and potential of event-driven gait analysis.
  arXiv  Detail & Related papers  (2026-05-21T08:12:04Z)
• Spatial Orthogonal Refinement for Robust RGB-Event Visual Object Tracking [8.019596736149834]
  We propose SOR-Track, a framework for robust RGB-Event tracking based on Spatial Orthogonal Refinement (SOR)<n>Experiments on the large-scale FE108 benchmark demonstrate that SOR-Track consistently outperforms existing fusion-based trackers.<n>Despite its simplicity, the proposed method offers a principled and physics-grounded approach to multi-modal feature alignment and texture rectification.
  arXiv  Detail & Related papers  (2026-03-29T23:54:31Z)
• Learning Progressive Adaptation for Multi-Modal Tracking [67.50696675353451]
  Multi-modal trackers are typically built by adopting pre-trained RGB models with parameter-efficient fine-tuning modules.<n>To address the issues, we propose to perform Progressive Adaptation for Multi-Modal Tracking (PATrack)<n>This innovative approach incorporates modality-dependent, modality-entangled, and task-level adapters, effectively bridging the gap in adapting RGB pre-trained networks to multi-modal data.
  arXiv  Detail & Related papers  (2026-03-22T07:25:54Z)
• Decoupling Amplitude and Phase Attention in Frequency Domain for RGB-Event based Visual Object Tracking [51.31378940976401]
  Existing RGB-Event tracking approaches fail to fully exploit the unique advantages of event cameras.<n>We propose a novel tracking framework that performs early fusion in the frequency domain, enabling effective aggregation of high-frequency information from the event modality.<n>Experiments on three widely used RGB-Event tracking benchmark datasets, including FE108, FELT, and COESOT, demonstrate the high performance and efficiency of our method.
  arXiv  Detail & Related papers  (2026-01-03T01:10:17Z)
• CADTrack: Learning Contextual Aggregation with Deformable Alignment for Robust RGBT Tracking [68.71826342377004]
  RGB-Thermal (RGBT) tracking aims to exploit visible and thermal infrared modalities for robust all-weather object tracking.<n>Existing RGBT trackers struggle to resolve modality discrepancies, which poses great challenges for robust feature representation.<n>We propose a novel Contextual Aggregation with Deformable Alignment framework called CADTrack for RGBT Tracking.
  arXiv  Detail & Related papers  (2025-11-22T08:10:02Z)
• Mamba-FETrack V2: Revisiting State Space Model for Frame-Event based Visual Object Tracking [9.353589376846902]
  We propose an efficient RGB-Event object tracking framework based on the linear-complexity Vision Mamba network.<n>The source code and pre-trained models will be released at https://github.com/Event-AHU/Mamba_FETrack.
  arXiv  Detail & Related papers  (2025-06-30T12:24:01Z)
• Learning Flow-Guided Registration for RGB-Event Semantic Segmentation [22.996619370156584]
  Event cameras capture microsecond-level motion cues that complement RGB sensors.<n>We recast RGB-Event segmentation from fusion to registration.<n>We propose BRENet, a novel flow-guided bidirectional framework that adaptively matches correspondence between the asymmetric modalities.
  arXiv  Detail & Related papers  (2025-05-02T19:19:58Z)
• Spatially-guided Temporal Aggregation for Robust Event-RGB Optical Flow Estimation [47.75348821902489]
  Current optical flow methods exploit the stable appearance of frame (or RGB) data to establish robust correspondences across time.<n>Event cameras, on the other hand, provide high-temporal-resolution motion cues and excel in challenging scenarios.<n>This study introduces a novel approach that uses a spatially dense modality to guide the aggregation of the temporally dense event modality.
  arXiv  Detail & Related papers  (2025-01-01T13:40:09Z)
• VELoRA: A Low-Rank Adaptation Approach for Efficient RGB-Event based Recognition [54.27379947727035]
  This paper proposes a novel PEFT strategy to adapt the pre-trained foundation vision models for the RGB-Event-based classification.<n>The frame difference of the dual modalities is also considered to capture the motion cues via the frame difference backbone network.<n>The source code and pre-trained models will be released on urlhttps://github.com/Event-AHU/VELoRA.
  arXiv  Detail & Related papers  (2024-12-28T07:38:23Z)
• TENet: Targetness Entanglement Incorporating with Multi-Scale Pooling and Mutually-Guided Fusion for RGB-E Object Tracking [30.89375068036783]
  Existing approaches perform event feature extraction for RGB-E tracking using traditional appearance models. We propose an Event backbone (Pooler) to obtain a high-quality feature representation that is cognisant of the intrinsic characteristics of the event data. Our method significantly outperforms state-of-the-art trackers on two widely used RGB-E tracking datasets.
  arXiv  Detail & Related papers  (2024-05-08T12:19:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.