3D UAV Trajectory Estimation and Classification from Internet Videos via Language Model

• URL: http://arxiv.org/abs/2603.09070v1
• Date: Tue, 10 Mar 2026 01:29:17 GMT
• Title: 3D UAV Trajectory Estimation and Classification from Internet Videos via Language Model
• Abstract summary: We present a novel framework that derives UAV 3D trajectories and category information directly from Internet-scale UAV videos.<n>The proposed method closely approaches the current state-of-the-art, highlighting its robustness and applicability to real-world anti-UAV scenarios.
• Score: 15.346439463483991
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reliable 3D trajectory estimation of unmanned aerial vehicles (UAVs) is a fundamental requirement for anti-UAV systems, yet the acquisition of large-scale and accurately annotated trajectory data remains prohibitively expensive. In this work, we present a novel framework that derives UAV 3D trajectories and category information directly from Internet-scale UAV videos, without relying on manual annotations. First, language-driven data acquisition is employed to autonomously discover and collect UAV-related videos, while vision-language reasoning progressively filters task-relevant segments. Second, a training-free cross-modal label generation module is introduced to infer 3D trajectory hypotheses and UAV type cues. Third, a physics-informed refinement process is designed to impose temporal smoothness and kinematic consistency on the estimated trajectories. The resulting video clips and trajectory annotations can be readily utilized for downstream anti-UAV tasks. To assess effectiveness and generalization, we conduct zero-shot transfer experiments on a public, well-annotated 3D UAV benchmark. Results reveal a clear data scaling behavior: as the amount of online video data increases, zero-shot transfer performance on the target dataset improves consistently, without any target-domain training. The proposed method closely approaches the current state-of-the-art, highlighting its robustness and applicability to real-world anti-UAV scenarios. Code and datasets will be released upon acceptance.

Related papers

• DynFly: Dynamic-Aware Continuous Trajectory Generation for UAV Vision-Language Navigation in Urban Environments [55.73595346639892]
  We propose DynFly to bridge high-level navigation reasoning and executable UAV motion.<n>DynFly represents expert trajectories in B-spline control-point space and employs a Spline-DiT generator to learn conditional trajectory generation.<n>Experiments on the OpenUAV UAV-VLN benchmark show that DynFly improves both navigation performance and trajectory quality.
  arXiv  Detail & Related papers  (2026-06-30T13:33:48Z)
• SpatialUAV: Benchmarking Spatial Intelligence for Low-Altitude UAV Perception, Collaboration, and Motion [76.8430169655641]
  spatial intelligence is essential for low-altitude unmanned aerial vehicle (UAV) perception, collaboration, and navigation.<n>We introduce SpatialUAV, a real low-altitude UAV benchmark covering semantic discrimination, spatial relation, aerial-aerial collaboration, aerial-ground collaboration, and motion understanding.<n>We show that current models remain far from human-level performance, with pronounced bottlenecks in cross-view association, structured grounding, geometric reasoning, and temporal viewpoint understanding.
  arXiv  Detail & Related papers  (2026-06-26T09:16:55Z)
• Lifting Unlabeled Internet-level Data for 3D Scene Understanding [56.35787129098819]
  We demonstrate that carefully designed data engines can leverage web-curated, unlabeled videos to automatically generate training data.<n>We identify and analyze bottlenecks in automated data generation, revealing critical factors that determine the efficiency and effectiveness of learning from unlabeled data.
  arXiv  Detail & Related papers  (2026-04-02T11:26:44Z)
• How Far are Modern Trackers from UAV-Anti-UAV? A Million-Scale Benchmark and New Baseline [74.4054700050366]
  Unmanned Aerial Vehicles (UAVs) offer wide-ranging applications but also pose significant safety and privacy violation risks.<n>Current Anti-UAV research primarily focuses on RGB, infrared (IR), or RGB-IR videos captured by fixed ground cameras.<n>We propose a new multi-modal visual tracking task termed UAV-Anti-UAV, which involves a pursuer UAV tracking a target adversarial UAV in the video stream.
  arXiv  Detail & Related papers  (2025-12-08T10:19:54Z)
• Tracking the Unstable: Appearance-Guided Motion Modeling for Robust Multi-Object Tracking in UAV-Captured Videos [58.156141601478794]
  Multi-object tracking (UAVT) aims to track multiple objects while maintaining consistent identities across frames of a given video.<n>Existing methods typically model motion cues and appearance separately, overlooking their interplay and resulting in suboptimal tracking performance.<n>We propose AMOT, which exploits appearance and motion cues through two key components: an Appearance-Motion Consistency (AMC) matrix and a Motion-aware Track Continuation (MTC) module.
  arXiv  Detail & Related papers  (2025-08-03T12:06:47Z)
• Unsupervised UAV 3D Trajectories Estimation with Sparse Point Clouds [18.48877348628721]
  This paper presents a cost-effective, unsupervised UAV detection method using spatial-temporal sequence processing.<n>Our solution placed 4th in the CVPR 2024 UG2+ Challenge, demonstrating its practical effectiveness.<n>We plan to open-source all designs, code, and sample data for the research community.com/lianghanfang/UnLiDAR-UAV-Est.
  arXiv  Detail & Related papers  (2024-12-17T09:30:31Z)
• VECTOR: Velocity-Enhanced GRU Neural Network for Real-Time 3D UAV Trajectory Prediction [2.1825723033513165]
  We propose a new trajectory prediction method using Gated Recurrent Units (GRUs) within sequence-based neural networks. We employ both synthetic and real-world 3D UAV trajectory data, capturing a wide range of flight patterns, speeds, and agility. The GRU-based models significantly outperform state-of-the-art RNN approaches, with a mean square error (MSE) as low as 2 x 10-8.
  arXiv  Detail & Related papers  (2024-10-24T07:16:42Z)
• Clustering-based Learning for UAV Tracking and Pose Estimation [0.0]
  This work develops a clustering-based learning detection approach, CL-Det, for UAV tracking and pose estimation using two types of LiDARs. We first align the timestamps of Livox Avia data and LiDAR 360 data and then separate the point cloud of objects of interest (OOIs) from the environment. The proposed method shows competitive pose estimation performance and ranks 5th on the final leaderboard of the CVPR 2024 UG2+ Challenge.
  arXiv  Detail & Related papers  (2024-05-27T06:33:25Z)
• BEV-MAE: Bird's Eye View Masked Autoencoders for Point Cloud Pre-training in Autonomous Driving Scenarios [51.285561119993105]
  We present BEV-MAE, an efficient masked autoencoder pre-training framework for LiDAR-based 3D object detection in autonomous driving. Specifically, we propose a bird's eye view (BEV) guided masking strategy to guide the 3D encoder learning feature representation. We introduce a learnable point token to maintain a consistent receptive field size of the 3D encoder.
  arXiv  Detail & Related papers  (2022-12-12T08:15:03Z)
• Trajectory Design for UAV-Based Internet-of-Things Data Collection: A Deep Reinforcement Learning Approach [93.67588414950656]
  In this paper, we investigate an unmanned aerial vehicle (UAV)-assisted Internet-of-Things (IoT) system in a 3D environment. We present a TD3-based trajectory design for completion time minimization (TD3-TDCTM) algorithm. Our simulation results show the superiority of the proposed TD3-TDCTM algorithm over three conventional non-learning based baseline methods.
  arXiv  Detail & Related papers  (2021-07-23T03:33:29Z)
• Generating Synthetic Training Data for Deep Learning-Based UAV Trajectory Prediction [11.241614693184323]
  We present an approach for generating synthetic trajectory data of unmanned-aerial-vehicles (UAVs) in image space. We show that an RNN-based prediction model solely trained on the generated data can outperform classic reference models on a real-world UAV tracking dataset.
  arXiv  Detail & Related papers  (2021-07-01T13:08:31Z)
• Anti-UAV: A Large Multi-Modal Benchmark for UAV Tracking [59.06167734555191]
  Unmanned Aerial Vehicle (UAV) offers lots of applications in both commerce and recreation. We consider the task of tracking UAVs, providing rich information such as location and trajectory. We propose a dataset, Anti-UAV, with more than 300 video pairs containing over 580k manually annotated bounding boxes.
  arXiv  Detail & Related papers  (2021-01-21T07:00:15Z)
• AutoTrajectory: Label-free Trajectory Extraction and Prediction from Videos using Dynamic Points [92.91569287889203]
  We present a novel, label-free algorithm, AutoTrajectory, for trajectory extraction and prediction. To better capture the moving objects in videos, we introduce dynamic points. We aggregate dynamic points to instance points, which stand for moving objects such as pedestrians in videos.
  arXiv  Detail & Related papers  (2020-07-11T08:43:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.