Prompt-Driven Temporal Domain Adaptation for Nighttime UAV Tracking

• URL: http://arxiv.org/abs/2409.18533v1
• Date: Fri, 27 Sep 2024 08:12:28 GMT
• Title: Prompt-Driven Temporal Domain Adaptation for Nighttime UAV Tracking
• Authors: Changhong Fu, Yiheng Wang, Liangliang Yao, Guangze Zheng, Haobo Zuo, Jia Pan,
• Abstract summary: Nighttime UAV tracking under low-illuminated scenarios has achieved great progress by domain adaptation (DA) Previous DA training-based works are deficient in narrowing the discrepancy of temporal contexts for UAV trackers. This work proposes a prompt-driven temporal domain adaptation training framework to fully utilize temporal contexts for challenging nighttime UAV tracking.
• Score: 21.09039345888337
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Nighttime UAV tracking under low-illuminated scenarios has achieved great progress by domain adaptation (DA). However, previous DA training-based works are deficient in narrowing the discrepancy of temporal contexts for UAV trackers. To address the issue, this work proposes a prompt-driven temporal domain adaptation training framework to fully utilize temporal contexts for challenging nighttime UAV tracking, i.e., TDA. Specifically, the proposed framework aligns the distribution of temporal contexts from daytime and nighttime domains by training the temporal feature generator against the discriminator. The temporal-consistent discriminator progressively extracts shared domain-specific features to generate coherent domain discrimination results in the time series. Additionally, to obtain high-quality training samples, a prompt-driven object miner is employed to precisely locate objects in unannotated nighttime videos. Moreover, a new benchmark for long-term nighttime UAV tracking is constructed. Exhaustive evaluations on both public and self-constructed nighttime benchmarks demonstrate the remarkable performance of the tracker trained in TDA framework, i.e., TDA-Track. Real-world tests at nighttime also show its practicality. The code and demo videos are available at https://github.com/vision4robotics/TDA-Track.

Related papers

• DaDiff: Domain-aware Diffusion Model for Nighttime UAV Tracking [21.255323355862057]
  This work proposes a novel progressive alignment paradigm, named domain-aware diffusion model (DaDiff) DaDiff aligning nighttime LR object features to the daytime by virtue of progressive and stable generations. An elaborate nighttime UAV tracking benchmark is constructed for LR objects, consisting of 100 annotated sequences.
  arXiv  Detail & Related papers  (2024-10-16T06:14:15Z)
• STCMOT: Spatio-Temporal Cohesion Learning for UAV-Based Multiple Object Tracking [13.269416985959404]
  Multiple object tracking (MOT) in Unmanned Aerial Vehicle (UAV) videos is important for diverse applications in computer vision. We propose a novel Spatio-Temporal Cohesion Multiple Object Tracking framework (STCMOT) We use historical embedding features to model the representation of ReID and detection features in a sequential order. Our framework sets a new state-of-the-art performance in MOTA and IDF1 metrics.
  arXiv  Detail & Related papers  (2024-09-17T14:34:18Z)
• Temporal Correlation Meets Embedding: Towards a 2nd Generation of JDE-based Real-Time Multi-Object Tracking [52.04679257903805]
  Joint Detection and Embedding (JDE) trackers have demonstrated excellent performance in Multi-Object Tracking (MOT) tasks. Our tracker, named TCBTrack, achieves state-of-the-art performance on multiple public benchmarks.
  arXiv  Detail & Related papers  (2024-07-19T07:48:45Z)
• DCPT: Darkness Clue-Prompted Tracking in Nighttime UAVs [53.64523622330297]
  Existing nighttime unmanned aerial vehicle (UAV) trackers follow an "Enhance-then-Track" architecture. This separate enhancement and tracking fails to build an end-to-end trainable vision system. We propose Darkness Clue-Prompted Tracking (DCPT) that achieves robust UAV tracking at night by efficiently learning to generate darkness clue prompts.
  arXiv  Detail & Related papers  (2023-09-19T09:59:08Z)
• Towards Real-World Visual Tracking with Temporal Contexts [64.7981374129495]
  We propose a two-level framework (TCTrack) that can exploit temporal contexts efficiently. Based on it, we propose a stronger version for real-world visual tracking, i.e., TCTrack++. For feature extraction, we propose an attention-based temporally adaptive convolution to enhance the spatial features. For similarity map refinement, we introduce an adaptive temporal transformer to encode the temporal knowledge efficiently.
  arXiv  Detail & Related papers  (2023-08-20T17:59:40Z)
• SAM-DA: UAV Tracks Anything at Night with SAM-Powered Domain Adaptation [21.998189930406014]
  Domain adaptation (DA) has demonstrated significant promise for real-time nighttime unmanned aerial vehicle (UAV) tracking. The state-of-the-art (SOTA) DA still lacks the potential object with accurate pixel-level location and boundary to generate the high-quality target domain training sample. This work proposes a novel SAM-powered DA framework for real-time nighttime UAV tracking, i.e., SAM-DA.
  arXiv  Detail & Related papers  (2023-07-03T13:55:44Z)
• Tracker Meets Night: A Transformer Enhancer for UAV Tracking [20.74868878876137]
  spatial-channel Transformer-based low-light enhancer is proposed and plugged prior to tracking approaches. To achieve semantic-level low-light enhancement targeting the high-level task, novel spatial-channel attention module is proposed. In the enhancement process, SCT denoises and illuminates nighttime images simultaneously.
  arXiv  Detail & Related papers  (2023-03-20T09:18:52Z)
• Tracking Objects and Activities with Attention for Temporal Sentence Grounding [51.416914256782505]
  Temporal sentence (TSG) aims to localize the temporal segment which is semantically aligned with a natural language query in an untrimmed segment. We propose a novel Temporal Sentence Tracking Network (TSTNet), which contains (A) a Cross-modal Targets Generator to generate multi-modal and search space, and (B) a Temporal Sentence Tracker to track multi-modal targets' behavior and to predict query-related segment.
  arXiv  Detail & Related papers  (2023-02-21T16:42:52Z)
• Unsupervised Domain Adaptation for Nighttime Aerial Tracking [27.595253191781904]
  This work develops a novel unsupervised domain adaptation framework for nighttime aerial tracking. A unique object discovery approach is provided to generate training patches from raw nighttime tracking videos. With a Transformer day/night feature discriminator, the daytime tracking model is adversarially trained to track at night.
  arXiv  Detail & Related papers  (2022-03-20T12:35:09Z)
• Continuity-Discrimination Convolutional Neural Network for Visual Object Tracking [150.51667609413312]
  This paper proposes a novel model, named Continuity-Discrimination Convolutional Neural Network (CD-CNN) for visual object tracking. To address this problem, CD-CNN models temporal appearance continuity based on the idea of temporal slowness. In order to alleviate inaccurate target localization and drifting, we propose a novel notion, object-centroid.
  arXiv  Detail & Related papers  (2021-04-18T06:35:03Z)
• Robust Object Detection via Instance-Level Temporal Cycle Confusion [89.1027433760578]
  We study the effectiveness of auxiliary self-supervised tasks to improve the out-of-distribution generalization of object detectors. Inspired by the principle of maximum entropy, we introduce a novel self-supervised task, instance-level temporal cycle confusion (CycConf) For each object, the task is to find the most different object proposals in the adjacent frame in a video and then cycle back to itself for self-supervision.
  arXiv  Detail & Related papers  (2021-04-16T21:35:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.