DCPT: Darkness Clue-Prompted Tracking in Nighttime UAVs

• URL: http://arxiv.org/abs/2309.10491v4
• Date: Thu, 14 Mar 2024 12:06:10 GMT
• Title: DCPT: Darkness Clue-Prompted Tracking in Nighttime UAVs
• Authors: Jiawen Zhu, Huayi Tang, Zhi-Qi Cheng, Jun-Yan He, Bin Luo, Shihao Qiu, Shengming Li, Huchuan Lu,
• Abstract summary: Existing nighttime unmanned aerial vehicle (UAV) trackers follow an "Enhance-then-Track" architecture. This separate enhancement and tracking fails to build an end-to-end trainable vision system. We propose Darkness Clue-Prompted Tracking (DCPT) that achieves robust UAV tracking at night by efficiently learning to generate darkness clue prompts.
• Score: 53.64523622330297
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing nighttime unmanned aerial vehicle (UAV) trackers follow an "Enhance-then-Track" architecture - first using a light enhancer to brighten the nighttime video, then employing a daytime tracker to locate the object. This separate enhancement and tracking fails to build an end-to-end trainable vision system. To address this, we propose a novel architecture called Darkness Clue-Prompted Tracking (DCPT) that achieves robust UAV tracking at night by efficiently learning to generate darkness clue prompts. Without a separate enhancer, DCPT directly encodes anti-dark capabilities into prompts using a darkness clue prompter (DCP). Specifically, DCP iteratively learns emphasizing and undermining projections for darkness clues. It then injects these learned visual prompts into a daytime tracker with fixed parameters across transformer layers. Moreover, a gated feature aggregation mechanism enables adaptive fusion between prompts and between prompts and the base model. Extensive experiments show state-of-the-art performance for DCPT on multiple dark scenario benchmarks. The unified end-to-end learning of enhancement and tracking in DCPT enables a more trainable system. The darkness clue prompting efficiently injects anti-dark knowledge without extra modules. Code is available at https://github.com/bearyi26/DCPT.

Related papers

• Prompt-Driven Temporal Domain Adaptation for Nighttime UAV Tracking [21.09039345888337]
  Nighttime UAV tracking under low-illuminated scenarios has achieved great progress by domain adaptation (DA) Previous DA training-based works are deficient in narrowing the discrepancy of temporal contexts for UAV trackers. This work proposes a prompt-driven temporal domain adaptation training framework to fully utilize temporal contexts for challenging nighttime UAV tracking.
  arXiv  Detail & Related papers  (2024-09-27T08:12:28Z)
• Dense Optical Tracking: Connecting the Dots [82.79642869586587]
  DOT is a novel, simple and efficient method for solving the problem of point tracking in a video. We show that DOT is significantly more accurate than current optical flow techniques, outperforms sophisticated "universal trackers" like OmniMotion, and is on par with, or better than, the best point tracking algorithms like CoTracker.
  arXiv  Detail & Related papers  (2023-12-01T18:59:59Z)
• Disentangled Contrastive Image Translation for Nighttime Surveillance [87.03178320662592]
  Nighttime surveillance suffers from degradation due to poor illumination and arduous human annotations. Existing methods rely on multi-spectral images to perceive objects in the dark, which are troubled by low resolution and color absence. We argue that the ultimate solution for nighttime surveillance is night-to-day translation, or Night2Day. This paper contributes a new surveillance dataset called NightSuR. It includes six scenes to support the study on nighttime surveillance.
  arXiv  Detail & Related papers  (2023-07-11T06:40:27Z)
• Tracker Meets Night: A Transformer Enhancer for UAV Tracking [20.74868878876137]
  spatial-channel Transformer-based low-light enhancer is proposed and plugged prior to tracking approaches. To achieve semantic-level low-light enhancement targeting the high-level task, novel spatial-channel attention module is proposed. In the enhancement process, SCT denoises and illuminates nighttime images simultaneously.
  arXiv  Detail & Related papers  (2023-03-20T09:18:52Z)
• Unsupervised Domain Adaptation for Nighttime Aerial Tracking [27.595253191781904]
  This work develops a novel unsupervised domain adaptation framework for nighttime aerial tracking. A unique object discovery approach is provided to generate training patches from raw nighttime tracking videos. With a Transformer day/night feature discriminator, the daytime tracking model is adversarially trained to track at night.
  arXiv  Detail & Related papers  (2022-03-20T12:35:09Z)
• Efficient Visual Tracking with Exemplar Transformers [98.62550635320514]
  We introduce the Exemplar Transformer, an efficient transformer for real-time visual object tracking. E.T.Track, our visual tracker that incorporates Exemplar Transformer layers, runs at 47 fps on a CPU. This is up to 8 times faster than other transformer-based models.
  arXiv  Detail & Related papers  (2021-12-17T18:57:54Z)
• DarkLighter: Light Up the Darkness for UAV Tracking [14.901582782711627]
  This work proposes a low-light image enhancer namely DarkLighter, which dedicates to alleviate the impact of poor illumination and noise. A lightweight map estimation network, i.e., ME-Net, is trained to efficiently estimate illumination maps and noise maps jointly. Experiments are conducted with several SOTA trackers on numerous UAV dark tracking scenes.
  arXiv  Detail & Related papers  (2021-07-30T01:37:24Z)
• ADTrack: Target-Aware Dual Filter Learning for Real-Time Anti-Dark UAV Tracking [17.80444543935194]
  The proposed method integrates an efficient and effective low-light image enhancer into a CF-based tracker. The target-aware mask can be applied to jointly train a target-focused filter that assists the context filter for robust tracking. The results have shown that ADTrack favorably outperforms other state-of-the-art trackers and achieves a real-time speed of 34 frames/s on a single CPU.
  arXiv  Detail & Related papers  (2021-06-04T14:05:24Z)
• IoU Attack: Towards Temporally Coherent Black-Box Adversarial Attack for Visual Object Tracking [70.14487738649373]
  Adrial attack arises due to the vulnerability of deep neural networks to perceive input samples injected with imperceptible perturbations. We propose a decision-based black-box attack method for visual object tracking. We validate the proposed IoU attack on state-of-the-art deep trackers.
  arXiv  Detail & Related papers  (2021-03-27T16:20:32Z)
• Robust Visual Object Tracking with Two-Stream Residual Convolutional Networks [62.836429958476735]
  We propose a Two-Stream Residual Convolutional Network (TS-RCN) for visual tracking. Our TS-RCN can be integrated with existing deep learning based visual trackers. To further improve the tracking performance, we adopt a "wider" residual network ResNeXt as its feature extraction backbone.
  arXiv  Detail & Related papers  (2020-05-13T19:05:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.