Mamba-FETrack V2: Revisiting State Space Model for Frame-Event based Visual Object Tracking

• URL: http://arxiv.org/abs/2506.23783v1
• Date: Mon, 30 Jun 2025 12:24:01 GMT
• Title: Mamba-FETrack V2: Revisiting State Space Model for Frame-Event based Visual Object Tracking
• Authors: Shiao Wang, Ju Huang, Qingchuan Ma, Jinfeng Gao, Chunyi Xu, Xiao Wang, Lan Chen, Bo Jiang,
• Abstract summary: We propose an efficient RGB-Event object tracking framework based on the linear-complexity Vision Mamba network.<n>The source code and pre-trained models will be released at https://github.com/Event-AHU/Mamba_FETrack.
• Score: 9.353589376846902
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Combining traditional RGB cameras with bio-inspired event cameras for robust object tracking has garnered increasing attention in recent years. However, most existing multimodal tracking algorithms depend heavily on high-complexity Vision Transformer architectures for feature extraction and fusion across modalities. This not only leads to substantial computational overhead but also limits the effectiveness of cross-modal interactions. In this paper, we propose an efficient RGB-Event object tracking framework based on the linear-complexity Vision Mamba network, termed Mamba-FETrack V2. Specifically, we first design a lightweight Prompt Generator that utilizes embedded features from each modality, together with a shared prompt pool, to dynamically generate modality-specific learnable prompt vectors. These prompts, along with the modality-specific embedded features, are then fed into a Vision Mamba-based FEMamba backbone, which facilitates prompt-guided feature extraction, cross-modal interaction, and fusion in a unified manner. Finally, the fused representations are passed to the tracking head for accurate target localization. Extensive experimental evaluations on multiple RGB-Event tracking benchmarks, including short-term COESOT dataset and long-term datasets, i.e., FE108 and FELT V2, demonstrate the superior performance and efficiency of the proposed tracking framework. The source code and pre-trained models will be released on https://github.com/Event-AHU/Mamba_FETrack

Related papers

• Visual and Memory Dual Adapter for Multi-Modal Object Tracking [34.406308400305385]
  We propose a novel visual and memory dual adapter (VMDA) to construct more robust representations for multi-modal tracking.<n>We develop a simple but effective visual adapter that adaptively transfers discriminative cues from auxiliary modality to dominant modality.<n>We also design the memory adapter inspired by the human memory mechanism, which stores global temporal cues and performs dynamic update and retrieval operations.
  arXiv  Detail & Related papers  (2025-06-30T15:38:26Z)
• TENet: Targetness Entanglement Incorporating with Multi-Scale Pooling and Mutually-Guided Fusion for RGB-E Object Tracking [30.89375068036783]
  Existing approaches perform event feature extraction for RGB-E tracking using traditional appearance models. We propose an Event backbone (Pooler) to obtain a high-quality feature representation that is cognisant of the intrinsic characteristics of the event data. Our method significantly outperforms state-of-the-art trackers on two widely used RGB-E tracking datasets.
  arXiv  Detail & Related papers  (2024-05-08T12:19:08Z)
• Mamba-FETrack: Frame-Event Tracking via State Space Model [14.610806117193116]
  This paper proposes a novel RGB-Event tracking framework, Mamba-FETrack, based on the State Space Model (SSM) Specifically, we adopt two modality-specific Mamba backbone networks to extract the features of RGB frames and Event streams. Extensive experiments on FELT and FE108 datasets fully validated the efficiency and effectiveness of our proposed tracker.
  arXiv  Detail & Related papers  (2024-04-28T13:12:49Z)
• Long-Term Visual Object Tracking with Event Cameras: An Associative Memory Augmented Tracker and A Benchmark Dataset [9.366068518600583]
  We propose a new long-term, large-scale frame-event visual object tracking dataset, termed FELT.<n>We also propose a novel Associative Memory Transformer based RGB-Event long-term visual tracker, termed AMTTrack.
  arXiv  Detail & Related papers  (2024-03-09T08:49:50Z)
• Bi-directional Adapter for Multi-modal Tracking [67.01179868400229]
  We propose a novel multi-modal visual prompt tracking model based on a universal bi-directional adapter. We develop a simple but effective light feature adapter to transfer modality-specific information from one modality to another. Our model achieves superior tracking performance in comparison with both the full fine-tuning methods and the prompt learning-based methods.
  arXiv  Detail & Related papers  (2023-12-17T05:27:31Z)
• Single-Model and Any-Modality for Video Object Tracking [85.83753760853142]
  We introduce Un-Track, a Unified Tracker of a single set of parameters for any modality. To handle any modality, our method learns their common latent space through low-rank factorization and reconstruction techniques. Our Un-Track achieves +8.1 absolute F-score gain, on the DepthTrack dataset, by introducing only +2.14 (over 21.50) GFLOPs with +6.6M (over 93M) parameters.
  arXiv  Detail & Related papers  (2023-11-27T14:17:41Z)
• SpikeMOT: Event-based Multi-Object Tracking with Sparse Motion Features [52.213656737672935]
  SpikeMOT is an event-based multi-object tracker. SpikeMOT uses spiking neural networks to extract sparsetemporal features from event streams associated with objects.
  arXiv  Detail & Related papers  (2023-09-29T05:13:43Z)
• Revisiting Color-Event based Tracking: A Unified Network, Dataset, and Metric [53.88188265943762]
  We propose a single-stage backbone network for Color-Event Unified Tracking (CEUTrack), which achieves the above functions simultaneously. Our proposed CEUTrack is simple, effective, and efficient, which achieves over 75 FPS and new SOTA performance.
  arXiv  Detail & Related papers  (2022-11-20T16:01:31Z)
• RGB-Event Fusion for Moving Object Detection in Autonomous Driving [3.5397758597664306]
  Moving Object Detection (MOD) is a critical vision task for successfully achieving safe autonomous driving. Recent advances in sensor technologies, especially the Event camera, can naturally complement the conventional camera approach to better model moving objects. We propose RENet, a novel RGB-Event fusion Network, that jointly exploits the two complementary modalities to achieve more robust MOD.
  arXiv  Detail & Related papers  (2022-09-17T12:59:08Z)
• Joint Spatial-Temporal and Appearance Modeling with Transformer for Multiple Object Tracking [59.79252390626194]
  We propose a novel solution named TransSTAM, which leverages Transformer to model both the appearance features of each object and the spatial-temporal relationships among objects. The proposed method is evaluated on multiple public benchmarks including MOT16, MOT17, and MOT20, and it achieves a clear performance improvement in both IDF1 and HOTA.
  arXiv  Detail & Related papers  (2022-05-31T01:19:18Z)
• Visible-Thermal UAV Tracking: A Large-Scale Benchmark and New Baseline [80.13652104204691]
  In this paper, we construct a large-scale benchmark with high diversity for visible-thermal UAV tracking (VTUAV) We provide a coarse-to-fine attribute annotation, where frame-level attributes are provided to exploit the potential of challenge-specific trackers. In addition, we design a new RGB-T baseline, named Hierarchical Multi-modal Fusion Tracker (HMFT), which fuses RGB-T data in various levels.
  arXiv  Detail & Related papers  (2022-04-08T15:22:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.