Learn by Observation: Imitation Learning for Drone Patrolling from Videos of A Human Navigator

• URL: http://arxiv.org/abs/2008.13193v1
• Date: Sun, 30 Aug 2020 15:20:40 GMT
• Title: Learn by Observation: Imitation Learning for Drone Patrolling from Videos of A Human Navigator
• Authors: Yue Fan, Shilei Chu, Wei Zhang, Ran Song, and Yibin Li
• Abstract summary: We propose to let the drone learn patrolling in the air by observing and imitating how a human navigator does it on the ground. The observation process enables the automatic collection and annotation of data using inter-frame geometric consistency. A newly designed neural network is trained based on the annotated data to predict appropriate directions and translations.
• Score: 22.06785798356346
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present an imitation learning method for autonomous drone patrolling based only on raw videos. Different from previous methods, we propose to let the drone learn patrolling in the air by observing and imitating how a human navigator does it on the ground. The observation process enables the automatic collection and annotation of data using inter-frame geometric consistency, resulting in less manual effort and high accuracy. Then a newly designed neural network is trained based on the annotated data to predict appropriate directions and translations for the drone to patrol in a lane-keeping manner as humans. Our method allows the drone to fly at a high altitude with a broad view and low risk. It can also detect all accessible directions at crossroads and further carry out the integration of available user instructions and autonomous patrolling control commands. Extensive experiments are conducted to demonstrate the accuracy of the proposed imitating learning process as well as the reliability of the holistic system for autonomous drone navigation. The codes, datasets as well as video demonstrations are available at https://vsislab.github.io/uavpatrol

Related papers

• Drone Detection and Tracking with YOLO and a Rule-based Method [0.0]
  An increased volume of drone activity in public spaces requires regulatory actions for purposes of privacy protection and safety. detection tasks are usually automated and performed by deep learning models which are trained on annotated image datasets. This paper builds on a previous work and extends an already published open source dataset. Since the detection models are based on a single image input, a simple cross-correlation based tracker is used to reduce detection drops and improve tracking performance in videos.
  arXiv  Detail & Related papers  (2025-02-07T19:53:10Z)
• A Cross-Scene Benchmark for Open-World Drone Active Tracking [54.235808061746525]
  Drone Visual Active Tracking aims to autonomously follow a target object by controlling the motion system based on visual observations. We propose a unified cross-scene cross-domain benchmark for open-world drone active tracking called DAT. We also propose a reinforcement learning-based drone tracking method called R-VAT.
  arXiv  Detail & Related papers  (2024-12-01T09:37:46Z)
• Chasing the Intruder: A Reinforcement Learning Approach for Tracking Intruder Drones [0.08192907805418582]
  We propose a reinforcement learning based approach for identifying and tracking any intruder drone using a chaser drone. Our proposed solution uses computer vision techniques interleaved with the policy learning framework of reinforcement learning. The results show that the reinforcement learning based policy converges to identify and track the intruder drone.
  arXiv  Detail & Related papers  (2023-09-10T16:31:40Z)
• AZTR: Aerial Video Action Recognition with Auto Zoom and Temporal Reasoning [63.628195002143734]
  We propose a novel approach for aerial video action recognition. Our method is designed for videos captured using UAVs and can run on edge or mobile devices. We present a learning-based approach that uses customized auto zoom to automatically identify the human target and scale it appropriately.
  arXiv  Detail & Related papers  (2023-03-02T21:24:19Z)
• TransVisDrone: Spatio-Temporal Transformer for Vision-based Drone-to-Drone Detection in Aerial Videos [57.92385818430939]
  Drone-to-drone detection using visual feed has crucial applications, such as detecting drone collisions, detecting drone attacks, or coordinating flight with other drones. Existing methods are computationally costly, follow non-end-to-end optimization, and have complex multi-stage pipelines, making them less suitable for real-time deployment on edge devices. We propose a simple yet effective framework, itTransVisDrone, that provides an end-to-end solution with higher computational efficiency.
  arXiv  Detail & Related papers  (2022-10-16T03:05:13Z)
• Aerial Vision-and-Dialog Navigation [10.596163697911525]
  We introduce Aerial Vision-and-Dialog Navigation (AVDN), to navigate a drone via natural language conversation. We build a drone simulator with a continuous environment and collect a new AVDN dataset of over 3k recorded navigation trajectories. We propose an effective Human Attention Aided Transformer model (HAA-Transformer) which learns to predict both navigation waypoints and human attention.
  arXiv  Detail & Related papers  (2022-05-24T17:28:14Z)
• Scarce Data Driven Deep Learning of Drones via Generalized Data Distribution Space [12.377024173799631]
  We show how understanding the general distribution of the drone data via a Generative Adversarial Network (GAN) can allow us to acquire missing data to achieve rapid and more accurate learning. We demonstrate our results on a drone image dataset, which contains both real drone images as well as simulated images from computer-aided design.
  arXiv  Detail & Related papers  (2021-08-18T17:07:32Z)
• Dogfight: Detecting Drones from Drones Videos [58.158988162743825]
  This paper attempts to address the problem of drones detection from other flying drones variations. The erratic movement of the source and target drones, small size, arbitrary shape, large intensity, and occlusion make this problem quite challenging. To handle this, instead of using region-proposal based methods, we propose to use a two-stage segmentation-based approach.
  arXiv  Detail & Related papers  (2021-03-31T17:43:31Z)
• ViNG: Learning Open-World Navigation with Visual Goals [82.84193221280216]
  We propose a learning-based navigation system for reaching visually indicated goals. We show that our system, which we call ViNG, outperforms previously-proposed methods for goal-conditioned reinforcement learning. We demonstrate ViNG on a number of real-world applications, such as last-mile delivery and warehouse inspection.
  arXiv  Detail & Related papers  (2020-12-17T18:22:32Z)
• Vision-based Drone Flocking in Outdoor Environments [9.184987303791292]
  This letter proposes a vision-based detection and tracking algorithm for drone swarms. We employ a convolutional neural network to detect and localize nearby agents onboard the quadcopters in real-time. We show that the drones can safely navigate in an outdoor environment despite substantial background clutter and difficult lighting conditions.
  arXiv  Detail & Related papers  (2020-12-02T14:44:40Z)
• LaND: Learning to Navigate from Disengagements [158.6392333480079]
  We present a reinforcement learning approach for learning to navigate from disengagements, or LaND. LaND learns a neural network model that predicts which actions lead to disengagements given the current sensory observation, and then at test time plans and executes actions that avoid disengagements. Our results demonstrate LaND can successfully learn to navigate in diverse, real world sidewalk environments, outperforming both imitation learning and reinforcement learning approaches.
  arXiv  Detail & Related papers  (2020-10-09T17:21:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.