Instance-level Visual Active Tracking with Occlusion-Aware Planning

• URL: http://arxiv.org/abs/2604.21453v1
• Date: Thu, 23 Apr 2026 09:11:50 GMT
• Title: Instance-level Visual Active Tracking with Occlusion-Aware Planning
• Abstract summary: Visual Active Tracking (VAT) aims to control cameras to follow a target in 3D space.<n>VAT faces two key bottlenecks in real-world deployment: confusion from visually similar distractors and severe failure under occlusions.<n>We propose OA-VAT, a unified pipeline with three complementary modules.
• Score: 61.982298426203165
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Visual Active Tracking (VAT) aims to control cameras to follow a target in 3D space, which is critical for applications like drone navigation and security surveillance. However, it faces two key bottlenecks in real-world deployment: confusion from visually similar distractors caused by insufficient instance-level discrimination and severe failure under occlusions due to the absence of active planning. To address these, we propose OA-VAT, a unified pipeline with three complementary modules. First, a training-free Instance-Aware Offline Prototype Initialization aggregates multi-view augmented features via DINOv3 to construct discriminative instance prototypes, mitigating distractor confusion. Second, an Online Prototype Enhancement Tracker enhances prototypes online and integrates a confidence-aware Kalman filter for stable tracking under appearance and motion changes. Third, an Occlusion-Aware Trajectory Planner, trained on our new Planning-20k dataset, uses conditional diffusion to generate obstacle-avoiding paths for occlusion recovery. Experiments demonstrate OA-VAT achieves 0.93 average SR on UnrealCV (+2.2% vs. SOTA TrackVLA), 90.8% average CAR on real-world datasets (+12.1% vs. SOTA GC-VAT), and 81.6% TSR on a DJI Tello drone. Running at 35 FPS on an RTX 3090, it delivers robust, real-time performance for practical deployment.

Related papers

• Embodied-Navigator: Point, Think, Memorize, and Align for Efficient Navigation [68.02899606570223]
  We propose TAMP-Nav, a unified framework for efficient embodied navigation.<n>First, we introduce a Pixel-to-3D Action Formulation (Point) that reformulates navigation into 2D visual coordinates.<n>Second, we propose an integrated Selective Reasoning and Anchor-Trajectory Memory mechanism (Think and Memorize)
  arXiv  Detail & Related papers  (2026-08-18T08:37:08Z)
• CosFly-VLA: A Spatially Aware Vision-Language-Action Model for UAV Tracking [6.465600276213322]
  Vision-Language-Action (VLA) policies can track visible targets effectively, but their performance often degrades when buildings, vegetation, or roadside objects block the line of sight.<n>We present CosFly-VLA, a spatially aware VLA model that jointly grounds the target, estimates its visibility, and generates continuous flight actions.<n>Relative to OpenVLA, CosFly-VLA-0.8B reduces open-loop Average Displacement Error (ADE) by 34.1% on seen-test and 35.3% on unseen-test.
  arXiv  Detail & Related papers  (2026-07-16T13:52:08Z)
• Time-to-Collision Based Dynamic Obstacle Avoidance Using Pretrained Vision Models for Robots in Unstructured Environments [12.499778437707759]
  We present a vision-based dynamic obstacle avoidance method that operates entirely on real-world data.<n>Our approach avoids the sim-to-real transfer problem inherent in simulation-trained policies.<n>It detects at least one frame with TTC less than 1 second for 20 out of 22 unique physical obstacles present in our test sequences.
  arXiv  Detail & Related papers  (2026-07-08T19:41:19Z)
• Fly360: Omnidirectional Obstacle Avoidance within Drone View [23.21774398877285]
  Obstacle avoidance in unmanned aerial vehicles (UAVs) has gained increasing attention with the growing focus on spatial intelligence.<n>Current obstacle-avoidance methods mainly depend on limited field-of-view sensors and are ill-suited for UAV scenarios.<n>This limitation motivates us to explore omnidirectional obstacle avoidance for panoramic drones with full-view perception.
  arXiv  Detail & Related papers  (2026-03-06T18:59:43Z)
• YOLOv8-SMOT: An Efficient and Robust Framework for Real-Time Small Object Tracking via Slice-Assisted Training and Adaptive Association [5.07987775511372]
  This paper details our championship-winning solution in the MVA 2025 "Finding Birds" Small Multi-Object Tracking Challenge (SMOT4SB)<n>It adopts the tracking-by-detection paradigm with targeted innovations at both the detection and association levels.<n>Our method achieves state-of-the-art performance on the SMOT4SB public test set, reaching an SO-HOTA score of textbf55.205.
  arXiv  Detail & Related papers  (2025-07-16T09:51:19Z)
• Open-World Drone Active Tracking with Goal-Centered Rewards [62.21394499788672]
  Drone Visual Active Tracking aims to autonomously follow a target object by controlling the motion system based on visual observations.<n>We propose DAT, the first open-world drone active air-to-ground tracking benchmark.<n>We also propose GC-VAT, which aims to improve the performance of drone tracking targets in complex scenarios.
  arXiv  Detail & Related papers  (2024-12-01T09:37:46Z)
• ODTFormer: Efficient Obstacle Detection and Tracking with Stereo Cameras Based on Transformer [12.58804521609764]
  ODTFormer is a Transformer-based model to address both obstacle detection and tracking problems. We report comparable accuracy to state-of-the-art obstacle tracking models while requiring only a fraction of their cost.
  arXiv  Detail & Related papers  (2024-03-21T17:59:55Z)
• BEVTrack: A Simple and Strong Baseline for 3D Single Object Tracking in Bird's-Eye View [54.48052449493636]
  3D Single Object Tracking (SOT) is a fundamental task in computer vision and plays a critical role in applications like autonomous driving.<n>We propose BEVTrack, a simple yet effective motion-based tracking method.<n>We show that BEVTrack achieves state-of-the-art results while operating at 200 FPS, enabling real-time applicability.
  arXiv  Detail & Related papers  (2023-09-05T12:42:26Z)
• An Effective Motion-Centric Paradigm for 3D Single Object Tracking in Point Clouds [50.19288542498838]
  3D single object tracking in LiDAR point clouds (LiDAR SOT) plays a crucial role in autonomous driving. Current approaches all follow the Siamese paradigm based on appearance matching. We introduce a motion-centric paradigm to handle LiDAR SOT from a new perspective.
  arXiv  Detail & Related papers  (2023-03-21T17:28:44Z)
• VISTA: Boosting 3D Object Detection via Dual Cross-VIew SpaTial Attention [32.44687996180621]
  We propose to adaptively fuse multi-view features in a global spatial context via Dual Cross-VIew SpaTial Attention (VISTA) The proposed VISTA is a novel plug-and-play fusion module, wherein the multi-layer perceptron widely adopted in standard attention modules is replaced with a convolutional one. At the time of submission, our method achieves 63.0% in overall mAP and 69.8% in NDS on the nuScenes benchmark, outperforming all published methods by up to 24% in safety-crucial categories such as cyclist.
  arXiv  Detail & Related papers  (2022-03-18T02:34:59Z)
• Cascaded Regression Tracking: Towards Online Hard Distractor Discrimination [202.2562153608092]
  We propose a cascaded regression tracker with two sequential stages. In the first stage, we filter out abundant easily-identified negative candidates. In the second stage, a discrete sampling based ridge regression is designed to double-check the remaining ambiguous hard samples.
  arXiv  Detail & Related papers  (2020-06-18T07:48:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.