BTMTrack: Robust RGB-T Tracking via Dual-template Bridging and Temporal-Modal Candidate Elimination

• URL: http://arxiv.org/abs/2501.03616v3
• Date: Sat, 18 Jan 2025 17:56:44 GMT
• Title: BTMTrack: Robust RGB-T Tracking via Dual-template Bridging and Temporal-Modal Candidate Elimination
• Authors: Zhongxuan Zhang, Bi Zeng, Xinyu Ni, Yimin Du,
• Abstract summary: We propose BTMTrack, a novel framework for RGB-T tracking. The core of our approach lies in the dual-template backbone network and the Temporal-Modal Candidate Elimination (TMCE) strategy. Our method achieves state-of-the-art performance, with a 72.3% precision rate on the LasHeR test set and competitive results on RGBT210 and RGBT234 datasets.
• Score: 0.16249398255272313
• License:
• Abstract: RGB-T tracking leverages the complementary strengths of RGB and thermal infrared (TIR) modalities to address challenging scenarios such as low illumination and adverse weather. However, existing methods often fail to effectively integrate temporal information and perform efficient cross-modal interactions, which constrain their adaptability to dynamic targets. In this paper, we propose BTMTrack, a novel framework for RGB-T tracking. The core of our approach lies in the dual-template backbone network and the Temporal-Modal Candidate Elimination (TMCE) strategy. The dual-template backbone effectively integrates temporal information, while the TMCE strategy focuses the model on target-relevant tokens by evaluating temporal and modal correlations, reducing computational overhead and avoiding irrelevant background noise. Building upon this foundation, we propose the Temporal Dual Template Bridging (TDTB) module, which facilitates precise cross-modal fusion through dynamically filtered tokens. This approach further strengthens the interaction between templates and the search region. Extensive experiments conducted on three benchmark datasets demonstrate the effectiveness of BTMTrack. Our method achieves state-of-the-art performance, with a 72.3% precision rate on the LasHeR test set and competitive results on RGBT210 and RGBT234 datasets.

Related papers

• Exploiting Multimodal Spatial-temporal Patterns for Video Object Tracking [53.33637391723555]
  We propose a unified multimodal spatial-temporal tracking approach named STTrack. In contrast to previous paradigms, we introduced a temporal state generator (TSG) that continuously generates a sequence of tokens containing multimodal temporal information. These temporal information tokens are used to guide the localization of the target in the next time state, establish long-range contextual relationships between video frames, and capture the temporal trajectory of the target.
  arXiv  Detail & Related papers  (2024-12-20T09:10:17Z)
• Divide-and-Conquer: Confluent Triple-Flow Network for RGB-T Salient Object Detection [70.84835546732738]
  RGB-Thermal Salient Object Detection aims to pinpoint prominent objects within aligned pairs of visible and thermal infrared images. Traditional encoder-decoder architectures may not have adequately considered the robustness against noise originating from defective modalities. We propose the ConTriNet, a robust Confluent Triple-Flow Network employing a Divide-and-Conquer strategy.
  arXiv  Detail & Related papers  (2024-12-02T14:44:39Z)
• Cross Fusion RGB-T Tracking with Bi-directional Adapter [8.425592063392857]
  We propose a novel Cross Fusion RGB-T Tracking architecture (CFBT) The effectiveness of CFBT relies on three newly designed cross-temporal information fusion modules. Experiments on three popular RGB-T tracking benchmarks demonstrate that our method achieves new state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-08-30T02:45:56Z)
• Unified Single-Stage Transformer Network for Efficient RGB-T Tracking [47.88113335927079]
  We propose a single-stage Transformer RGB-T tracking network, namely USTrack, which unifies the above three stages into a single ViT (Vision Transformer) backbone. With this structure, the network can extract fusion features of the template and search region under the mutual interaction of modalities. Experiments on three popular RGB-T tracking benchmarks demonstrate that our method achieves new state-of-the-art performance while maintaining the fastest inference speed 84.2FPS.
  arXiv  Detail & Related papers  (2023-08-26T05:09:57Z)
• RGB-T Tracking Based on Mixed Attention [5.151994214135177]
  RGB-T tracking involves the use of images from both visible and thermal modalities. An RGB-T tracker based on mixed attention mechanism to achieve complementary fusion of modalities is proposed in this paper.
  arXiv  Detail & Related papers  (2023-04-09T15:59:41Z)
• Ret3D: Rethinking Object Relations for Efficient 3D Object Detection in Driving Scenes [82.4186966781934]
  We introduce a simple, efficient, and effective two-stage detector, termed as Ret3D. At the core of Ret3D is the utilization of novel intra-frame and inter-frame relation modules. With negligible extra overhead, Ret3D achieves the state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-08-18T03:48:58Z)
• Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Framework [76.70603443624012]
  We propose a novel one-stream tracking (OSTrack) framework that unifies feature learning and relation modeling. In this way, discriminative target-oriented features can be dynamically extracted by mutual guidance. OSTrack achieves state-of-the-art performance on multiple benchmarks, in particular, it shows impressive results on the one-shot tracking benchmark GOT-10k.
  arXiv  Detail & Related papers  (2022-03-22T18:37:11Z)
• Temporal Aggregation for Adaptive RGBT Tracking [14.00078027541162]
  We propose an RGBT tracker which takes clues into account for robust appearance model learning. Unlike most existing RGBT trackers that implement object tracking tasks with only spatial information included, temporal information is further considered in this method.
  arXiv  Detail & Related papers  (2022-01-22T02:31:56Z)
• Jointly Modeling Motion and Appearance Cues for Robust RGB-T Tracking [85.333260415532]
  We develop a novel late fusion method to infer the fusion weight maps of both RGB and thermal (T) modalities. When the appearance cue is unreliable, we take motion cues into account to make the tracker robust. Numerous results on three recent RGB-T tracking datasets show that the proposed tracker performs significantly better than other state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2020-07-04T08:11:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.