WildLive: Near Real-time Visual Wildlife Tracking onboard UAVs

• URL: http://arxiv.org/abs/2504.10165v2
• Date: Tue, 15 Apr 2025 12:06:09 GMT
• Title: WildLive: Near Real-time Visual Wildlife Tracking onboard UAVs
• Authors: Nguyen Ngoc Dat, Tom Richardson, Matthew Watson, Kilian Meier, Jenna Kline, Sid Reid, Guy Maalouf, Duncan Hine, Majid Mirmehdi, Tilo Burghardt,
• Abstract summary: We introduce WildLive -- a near real-time animal detection and tracking framework for high-resolution imagery running directly onboard aerial vehicles (UAVs)<n>The system performs multi-animal detection tracking at 17 fps+ for HD and 7 fps+ on 4K video streams suitable for operation during higher altitude flights to minimise animal disturbance.<n>We introduce our WildLive dataset, which comprises 200k+ annotated animal instances across 19k+ frames from 4K UAV videos collected at the Ol Pejeta Conservancy in Kenya.
• Score: 4.215854427679142
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Live tracking of wildlife via high-resolution video processing directly onboard drones is widely unexplored and most existing solutions rely on streaming video to ground stations to support navigation. Yet, both autonomous animal-reactive flight control beyond visual line of sight and/or mission-specific individual and behaviour recognition tasks rely to some degree on this capability. In response, we introduce WildLive -- a near real-time animal detection and tracking framework for high-resolution imagery running directly onboard uncrewed aerial vehicles (UAVs). The system performs multi-animal detection and tracking at 17fps+ for HD and 7fps+ on 4K video streams suitable for operation during higher altitude flights to minimise animal disturbance. Our system is optimised for Jetson Orin AGX onboard hardware. It integrates the efficiency of sparse optical flow tracking and mission-specific sampling with device-optimised and proven YOLO-driven object detection and segmentation techniques. Essentially, computational resource is focused onto spatio-temporal regions of high uncertainty to significantly improve UAV processing speeds without domain-specific loss of accuracy. Alongside, we introduce our WildLive dataset, which comprises 200k+ annotated animal instances across 19k+ frames from 4K UAV videos collected at the Ol Pejeta Conservancy in Kenya. All frames contain ground truth bounding boxes, segmentation masks, as well as individual tracklets and tracking point trajectories. We compare our system against current object tracking approaches including OC-SORT, ByteTrack, and SORT. Our materials are available at: https://dat-nguyenvn.github.io/WildLive/

Related papers

• Context-Aware Outlier Rejection for Robust Multi-View 3D Tracking of Similar Small Birds in An Outdoor Aviary [39.35431651202991]
  This paper presents a novel approach for robust 3D tracking of multiple birds in an outdoor aviary using a multi-camera system. Our method addresses the challenges of visually similar birds and their rapid movements by leveraging environmental landmarks for enhanced feature matching and 3D reconstruction. We also provide a large annotated dataset of 80 birds residing in four enclosures for 20 hours of footage which provides a rich testbed for researchers in computer vision, ornithologists, and ecologists.
  arXiv  Detail & Related papers  (2024-12-21T07:20:57Z)
• A Cross-Scene Benchmark for Open-World Drone Active Tracking [54.235808061746525]
  Drone Visual Active Tracking aims to autonomously follow a target object by controlling the motion system based on visual observations.<n>We propose a unified cross-scene cross-domain benchmark for open-world drone active tracking called DAT.<n>We also propose a reinforcement learning-based drone tracking method called R-VAT.
  arXiv  Detail & Related papers  (2024-12-01T09:37:46Z)
• Towards Real-Time Fast Unmanned Aerial Vehicle Detection Using Dynamic Vision Sensors [6.03212980984729]
  Unmanned Aerial Vehicles (UAVs) are gaining popularity in civil and military applications. prevention and detection of UAVs are pivotal to guarantee confidentiality and safety. This paper presents F-UAV-D (Fast Unmanned Aerial Vehicle Detector), an embedded system that enables fast-moving drone detection.
  arXiv  Detail & Related papers  (2024-03-18T15:27:58Z)
• Multiview Aerial Visual Recognition (MAVREC): Can Multi-view Improve Aerial Visual Perception? [57.77643186237265]
  We present Multiview Aerial Visual RECognition or MAVREC, a video dataset where we record synchronized scenes from different perspectives. MAVREC consists of around 2.5 hours of industry-standard 2.7K resolution video sequences, more than 0.5 million frames, and 1.1 million annotated bounding boxes. This makes MAVREC the largest ground and aerial-view dataset, and the fourth largest among all drone-based datasets.
  arXiv  Detail & Related papers  (2023-12-07T18:59:14Z)
• BEVTrack: A Simple and Strong Baseline for 3D Single Object Tracking in Bird's-Eye View [56.77287041917277]
  3D Single Object Tracking (SOT) is a fundamental task of computer vision, proving essential for applications like autonomous driving. In this paper, we propose BEVTrack, a simple yet effective baseline method. By estimating the target motion in Bird's-Eye View (BEV) to perform tracking, BEVTrack demonstrates surprising simplicity from various aspects, i.e., network designs, training objectives, and tracking pipeline, while achieving superior performance.
  arXiv  Detail & Related papers  (2023-09-05T12:42:26Z)
• Improving Underwater Visual Tracking With a Large Scale Dataset and Image Enhancement [70.2429155741593]
  This paper presents a new dataset and general tracker enhancement method for Underwater Visual Object Tracking (UVOT) It poses distinct challenges; the underwater environment exhibits non-uniform lighting conditions, low visibility, lack of sharpness, low contrast, camouflage, and reflections from suspended particles. We propose a novel underwater image enhancement algorithm designed specifically to boost tracking quality. The method has resulted in a significant performance improvement, of up to 5.0% AUC, of state-of-the-art (SOTA) visual trackers.
  arXiv  Detail & Related papers  (2023-08-30T07:41:26Z)
• Propagate And Calibrate: Real-time Passive Non-line-of-sight Tracking [84.38335117043907]
  We propose a purely passive method to track a person walking in an invisible room by only observing a relay wall. To excavate imperceptible changes in videos of the relay wall, we introduce difference frames as an essential carrier of temporal-local motion messages. To evaluate the proposed method, we build and publish the first dynamic passive NLOS tracking dataset, NLOS-Track.
  arXiv  Detail & Related papers  (2023-03-21T12:18:57Z)
• Continuity-Aware Latent Interframe Information Mining for Reliable UAV Tracking [5.9397055042513465]
  Unmanned aerial vehicle (UAV) tracking is crucial for autonomous navigation and has broad applications in robotic automation fields. This work proposes a novel framework with continuity-aware latent interframe information mining for reliable UAV tracking, i.e., ClimRT.
  arXiv  Detail & Related papers  (2023-03-08T11:42:57Z)
• AVisT: A Benchmark for Visual Object Tracking in Adverse Visibility [125.77396380698639]
  AVisT is a benchmark for visual tracking in diverse scenarios with adverse visibility. AVisT comprises 120 challenging sequences with 80k annotated frames, spanning 18 diverse scenarios. We benchmark 17 popular and recent trackers on AVisT with detailed analysis of their tracking performance across attributes.
  arXiv  Detail & Related papers  (2022-08-14T17:49:37Z)
• Siamese Object Tracking for Unmanned Aerial Vehicle: A Review and Comprehensive Analysis [15.10348491862546]
  Unmanned aerial vehicle (UAV)-based visual object tracking has enabled a wide range of applications. Siamese networks shine in visual object tracking with their promising balance of accuracy, robustness, and speed.
  arXiv  Detail & Related papers  (2022-05-09T13:53:34Z)
• Scalable and Real-time Multi-Camera Vehicle Detection, Re-Identification, and Tracking [58.95210121654722]
  We propose a real-time city-scale multi-camera vehicle tracking system that handles real-world, low-resolution CCTV instead of idealized and curated video streams. Our method is ranked among the top five performers on the public leaderboard.
  arXiv  Detail & Related papers  (2022-04-15T12:47:01Z)
• Small or Far Away? Exploiting Deep Super-Resolution and Altitude Data for Aerial Animal Surveillance [3.8015092217142223]
  We show that a holistic attention network based super-resolution approach and a custom-built altitude data exploitation network can increase the detection efficacy in real-world settings. We evaluate the system on two public, large aerial-capture animal datasets, SAVMAP and AED.
  arXiv  Detail & Related papers  (2021-11-12T17:30:55Z)
• A Multi-UAV System for Exploration and Target Finding in Cluttered and GPS-Denied Environments [68.31522961125589]
  We propose a framework for a team of UAVs to cooperatively explore and find a target in complex GPS-denied environments with obstacles. The team of UAVs autonomously navigates, explores, detects, and finds the target in a cluttered environment with a known map. Results indicate that the proposed multi-UAV system has improvements in terms of time-cost, the proportion of search area surveyed, as well as successful rates for search and rescue missions.
  arXiv  Detail & Related papers  (2021-07-19T12:54:04Z)
• AcinoSet: A 3D Pose Estimation Dataset and Baseline Models for Cheetahs in the Wild [51.35013619649463]
  We present an extensive dataset of free-running cheetahs in the wild, called AcinoSet. The dataset contains 119,490 frames of multi-view synchronized high-speed video footage, camera calibration files and 7,588 human-annotated frames. The resulting 3D trajectories, human-checked 3D ground truth, and an interactive tool to inspect the data is also provided.
  arXiv  Detail & Related papers  (2021-03-24T15:54:11Z)
• Benchmarking Deep Trackers on Aerial Videos [5.414308305392762]
  In this paper, we compare ten trackers based on deep learning techniques on four aerial datasets. We choose top performing trackers utilizing different approaches, specifically tracking by detection, discriminative correlation filters, Siamese networks and reinforcement learning. Our findings indicate that the trackers perform significantly worse in aerial datasets compared to standard ground level videos.
  arXiv  Detail & Related papers  (2021-03-24T01:45:19Z)
• Correlation Filters for Unmanned Aerial Vehicle-Based Aerial Tracking: A Review and Experimental Evaluation [17.8941834997338]
  Discriminative correlation filter (DCF)-based trackers have stood out for their high computational efficiency and robustness on a single CPU. In this work, 23 state-of-the-art DCF-based trackers are summarized according to their innovations for solving various issues. Experiments show the performance, verify the feasibility, and demonstrate the current challenges of DCF-based trackers onboard UAV tracking.
  arXiv  Detail & Related papers  (2020-10-13T09:35:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.