WildLIFT: Lifting monocular drone video to 3D for species-agnostic wildlife monitoring

• URL: http://arxiv.org/abs/2604.24718v1
• Date: Mon, 27 Apr 2026 17:29:22 GMT
• Title: WildLIFT: Lifting monocular drone video to 3D for species-agnostic wildlife monitoring
• Abstract summary: We present WildLIFT, a framework that integrates 3D scene geometry from monocular drone video with open-vocabulary 2D instance segmentation.<n>We validate the framework on 2581, manually curated frames comprising over 6,700 3D detections across four large mammal species.
• Score: 2.8622185759617906
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Monocular RGB cameras mounted on drones are widely used for wildlife monitoring, yet most analytical pipelines remain confined to two-dimensional image space, leaving geometric information in video underexploited. We present WildLIFT, a computational framework that integrates three-dimensional scene geometry from monocular drone video with open-vocabulary 2D instance segmentation to enable species-agnostic 3D detection and tracking. Oriented 3D bounding box labels with semantic face information enable quantitative assessment of viewpoint coverage and inter-animal occlusion, producing structured metadata for downstream ecological analyses. We validate the framework on 2,581 manually curated frames comprising over 6,700 3D detections across four large mammal species. WildLIFT maintains high identity consistency in multi-animal scenes and substantially reduces manual 3D annotation effort through keyframe-based refinement. By transforming standard drone footage into structured 3D and viewpoint-aware representations, WildLIFT extends the analytical utility of aerial wildlife datasets for behavioural research and population monitoring.

Related papers

• BEAST3D: Animal behavioral analysis and neural encoding from multi-view video via Gaussian splatting [6.870390707295292]
  BEAST3D is a self-supervised pretraining framework that learns 3D visual representations from unlabeled, calibrated multi-view video.<n>BEAST3D reconstructs 3D structure with as few as four views by conditioning directly on known camera parameters.
  arXiv  Detail & Related papers  (2026-06-01T22:34:14Z)
• Adding Another Dimension to Image-based Animal Detection [3.0707196190091532]
  Detection algorithms lead to 2D bounding boxes that lack information about animal's orientation relative to the camera.<n>We present a pipeline that estimates 3D bounding boxes and projects them as robust labels into 2D image space.<n>We evaluate our method on the Animal3D dataset, demonstrating accurate performance across species and settings.
  arXiv  Detail & Related papers  (2026-04-10T10:57:55Z)
• SpatialTrackerV2: 3D Point Tracking Made Easy [73.0350898700048]
  SpatialTrackerV2 is a feed-forward 3D point tracking method for monocular videos.<n>It decomposes world-space 3D motion into scene geometry, camera ego-motion, and pixel-wise object motion.<n>By learning geometry and motion jointly from such heterogeneous data, SpatialTrackerV2 outperforms existing 3D tracking methods by 30%.
  arXiv  Detail & Related papers  (2025-07-16T17:59:03Z)
• MCBLT: Multi-Camera Multi-Object 3D Tracking in Long Videos [39.75692195719607]
  We propose a 3D object detection and tracking framework, named MCBLT.<n>Unlike existing methods, MCBLT has impressive generalizability across different scenes and diverse camera settings.<n>Our proposed MCBLT establishes a new state-of-the-art on the AICity'24 dataset with $81.22$ HOTA, and on the WildTrack dataset with $95.6$ IDF1.
  arXiv  Detail & Related papers  (2024-12-01T06:18:06Z)
• Virtual Pets: Animatable Animal Generation in 3D Scenes [84.0990909455833]
  We introduce Virtual Pet, a novel pipeline to model realistic and diverse motions for target animal species within a 3D environment. We leverage monocular internet videos and extract deformable NeRF representations for the foreground and static NeRF representations for the background. We develop a reconstruction strategy, encompassing species-level shared template learning and per-video fine-tuning.
  arXiv  Detail & Related papers  (2023-12-21T18:59:30Z)
• AutoDecoding Latent 3D Diffusion Models [95.7279510847827]
  We present a novel approach to the generation of static and articulated 3D assets that has a 3D autodecoder at its core. The 3D autodecoder framework embeds properties learned from the target dataset in the latent space. We then identify the appropriate intermediate volumetric latent space, and introduce robust normalization and de-normalization operations.
  arXiv  Detail & Related papers  (2023-07-07T17:59:14Z)
• 3D-POP -- An automated annotation approach to facilitate markerless 2D-3D tracking of freely moving birds with marker-based motion capture [1.1083289076967897]
  We propose a method that uses a motion capture (mo-cap) system to obtain a large amount of annotated data on animal movement and posture in a semi-automatic manner. Our method is novel in that it extracts the 3D positions of morphological keypoints in reference to the positions of markers attached to the animals. Using this method, we obtained, and offer here, a new dataset - 3D-POP with approximately 300k annotated frames (4 million instances) in the form of videos.
  arXiv  Detail & Related papers  (2023-03-23T11:03:18Z)
• Gait Recognition in the Wild with Dense 3D Representations and A Benchmark [86.68648536257588]
  Existing studies for gait recognition are dominated by 2D representations like the silhouette or skeleton of the human body in constrained scenes. This paper aims to explore dense 3D representations for gait recognition in the wild. We build the first large-scale 3D representation-based gait recognition dataset, named Gait3D.
  arXiv  Detail & Related papers  (2022-04-06T03:54:06Z)
• AcinoSet: A 3D Pose Estimation Dataset and Baseline Models for Cheetahs in the Wild [51.35013619649463]
  We present an extensive dataset of free-running cheetahs in the wild, called AcinoSet. The dataset contains 119,490 frames of multi-view synchronized high-speed video footage, camera calibration files and 7,588 human-annotated frames. The resulting 3D trajectories, human-checked 3D ground truth, and an interactive tool to inspect the data is also provided.
  arXiv  Detail & Related papers  (2021-03-24T15:54:11Z)
• Monocular Quasi-Dense 3D Object Tracking [99.51683944057191]
  A reliable and accurate 3D tracking framework is essential for predicting future locations of surrounding objects and planning the observer's actions in numerous applications such as autonomous driving. We propose a framework that can effectively associate moving objects over time and estimate their full 3D bounding box information from a sequence of 2D images captured on a moving platform.
  arXiv  Detail & Related papers  (2021-03-12T15:30:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.