Learn by Observation: Imitation Learning for Drone Patrolling from Videos of A Human Navigator

• URL: http://arxiv.org/abs/2008.13193v1
• Date: Sun, 30 Aug 2020 15:20:40 GMT
• Title: Learn by Observation: Imitation Learning for Drone Patrolling from Videos of A Human Navigator
• Authors: Yue Fan, Shilei Chu, Wei Zhang, Ran Song, and Yibin Li
• Abstract summary: We propose to let the drone learn patrolling in the air by observing and imitating how a human navigator does it on the ground. The observation process enables the automatic collection and annotation of data using inter-frame geometric consistency. A newly designed neural network is trained based on the annotated data to predict appropriate directions and translations.
• Score: 22.06785798356346
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present an imitation learning method for autonomous drone patrolling based only on raw videos. Different from previous methods, we propose to let the drone learn patrolling in the air by observing and imitating how a human navigator does it on the ground. The observation process enables the automatic collection and annotation of data using inter-frame geometric consistency, resulting in less manual effort and high accuracy. Then a newly designed neural network is trained based on the annotated data to predict appropriate directions and translations for the drone to patrol in a lane-keeping manner as humans. Our method allows the drone to fly at a high altitude with a broad view and low risk. It can also detect all accessible directions at crossroads and further carry out the integration of available user instructions and autonomous patrolling control commands. Extensive experiments are conducted to demonstrate the accuracy of the proposed imitating learning process as well as the reliability of the holistic system for autonomous drone navigation. The codes, datasets as well as video demonstrations are available at https://vsislab.github.io/uavpatrol

Related papers

• Chasing the Intruder: A Reinforcement Learning Approach for Tracking Intruder Drones [0.08192907805418582]
  We propose a reinforcement learning based approach for identifying and tracking any intruder drone using a chaser drone. Our proposed solution uses computer vision techniques interleaved with the policy learning framework of reinforcement learning. The results show that the reinforcement learning based policy converges to identify and track the intruder drone.
  arXiv  Detail & Related papers  (2023-09-10T16:31:40Z)
• Multi-model fusion for Aerial Vision and Dialog Navigation based on human attention aids [69.98258892165767]
  We present an aerial navigation task for the 2023 ICCV Conversation History. We propose an effective method of fusion training of Human Attention Aided Transformer model (HAA-Transformer) and Human Attention Aided LSTM (HAA-LSTM) models.
  arXiv  Detail & Related papers  (2023-08-27T10:32:52Z)
• AZTR: Aerial Video Action Recognition with Auto Zoom and Temporal Reasoning [63.628195002143734]
  We propose a novel approach for aerial video action recognition. Our method is designed for videos captured using UAVs and can run on edge or mobile devices. We present a learning-based approach that uses customized auto zoom to automatically identify the human target and scale it appropriately.
  arXiv  Detail & Related papers  (2023-03-02T21:24:19Z)
• TransVisDrone: Spatio-Temporal Transformer for Vision-based Drone-to-Drone Detection in Aerial Videos [57.92385818430939]
  Drone-to-drone detection using visual feed has crucial applications, such as detecting drone collisions, detecting drone attacks, or coordinating flight with other drones. Existing methods are computationally costly, follow non-end-to-end optimization, and have complex multi-stage pipelines, making them less suitable for real-time deployment on edge devices. We propose a simple yet effective framework, itTransVisDrone, that provides an end-to-end solution with higher computational efficiency.
  arXiv  Detail & Related papers  (2022-10-16T03:05:13Z)
• Aerial Vision-and-Dialog Navigation [10.596163697911525]
  We introduce Aerial Vision-and-Dialog Navigation (AVDN), to navigate a drone via natural language conversation. We build a drone simulator with a continuous environment and collect a new AVDN dataset of over 3k recorded navigation trajectories. We propose an effective Human Attention Aided Transformer model (HAA-Transformer) which learns to predict both navigation waypoints and human attention.
  arXiv  Detail & Related papers  (2022-05-24T17:28:14Z)
• Scarce Data Driven Deep Learning of Drones via Generalized Data Distribution Space [12.377024173799631]
  We show how understanding the general distribution of the drone data via a Generative Adversarial Network (GAN) can allow us to acquire missing data to achieve rapid and more accurate learning. We demonstrate our results on a drone image dataset, which contains both real drone images as well as simulated images from computer-aided design.
  arXiv  Detail & Related papers  (2021-08-18T17:07:32Z)
• Dogfight: Detecting Drones from Drones Videos [58.158988162743825]
  This paper attempts to address the problem of drones detection from other flying drones variations. The erratic movement of the source and target drones, small size, arbitrary shape, large intensity, and occlusion make this problem quite challenging. To handle this, instead of using region-proposal based methods, we propose to use a two-stage segmentation-based approach.
  arXiv  Detail & Related papers  (2021-03-31T17:43:31Z)
• ViNG: Learning Open-World Navigation with Visual Goals [82.84193221280216]
  We propose a learning-based navigation system for reaching visually indicated goals. We show that our system, which we call ViNG, outperforms previously-proposed methods for goal-conditioned reinforcement learning. We demonstrate ViNG on a number of real-world applications, such as last-mile delivery and warehouse inspection.
  arXiv  Detail & Related papers  (2020-12-17T18:22:32Z)
• Vision-based Drone Flocking in Outdoor Environments [9.184987303791292]
  This letter proposes a vision-based detection and tracking algorithm for drone swarms. We employ a convolutional neural network to detect and localize nearby agents onboard the quadcopters in real-time. We show that the drones can safely navigate in an outdoor environment despite substantial background clutter and difficult lighting conditions.
  arXiv  Detail & Related papers  (2020-12-02T14:44:40Z)
• LaND: Learning to Navigate from Disengagements [158.6392333480079]
  We present a reinforcement learning approach for learning to navigate from disengagements, or LaND. LaND learns a neural network model that predicts which actions lead to disengagements given the current sensory observation, and then at test time plans and executes actions that avoid disengagements. Our results demonstrate LaND can successfully learn to navigate in diverse, real world sidewalk environments, outperforming both imitation learning and reinforcement learning approaches.
  arXiv  Detail & Related papers  (2020-10-09T17:21:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.