Gesture Control of Micro-drone: A Lightweight-Net with Domain Randomization and Trajectory Generators

• URL: http://arxiv.org/abs/2301.12470v1
• Date: Sun, 29 Jan 2023 15:38:15 GMT
• Title: Gesture Control of Micro-drone: A Lightweight-Net with Domain Randomization and Trajectory Generators
• Authors: Isaac Osei Agyemang, Isaac Adjei Mensah, Sophyani Banaamwini Yussif, Fiasam Linda Delali, Bernard Cobinnah Mawuli, Bless Lord Y. Agbley, Collins Sey, and Joshua Berkohd
• Abstract summary: This study presents a computationally-efficient deep convolutional neural network that utilizes Gabor filters and spatial separable convolutions. The model aids a human operator in controlling a micro-drone via gestures. Using a low-cost DJI Tello drone for experiment verification, the computationally-efficient model shows promising results.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Micro-drones can be integrated into various industrial applications but are constrained by their computing power and expert pilots, a secondary challenge. This study presents a computationally-efficient deep convolutional neural network that utilizes Gabor filters and spatial separable convolutions with low computational complexities. An attention module is integrated with the model to complement the performance. Further, perception-based action space and trajectory generators are integrated with the model's predictions for intuitive navigation. The computationally-efficient model aids a human operator in controlling a micro-drone via gestures. Nearly 18% of computational resources are conserved using the NVIDIA GPU profiler during training. Using a low-cost DJI Tello drone for experiment verification, the computationally-efficient model shows promising results compared to a state-of-the-art and conventional computer vision-based technique.

Related papers

• Task-Oriented Real-time Visual Inference for IoVT Systems: A Co-design Framework of Neural Networks and Edge Deployment [61.20689382879937]
  Task-oriented edge computing addresses this by shifting data analysis to the edge. Existing methods struggle to balance high model performance with low resource consumption. We propose a novel co-design framework to optimize neural network architecture.
  arXiv  Detail & Related papers  (2024-10-29T19:02:54Z)
• Tiny-PULP-Dronets: Squeezing Neural Networks for Faster and Lighter Inference on Multi-Tasking Autonomous Nano-Drones [12.96119439129453]
  This work moves from PULP-Dronet, a State-of-the-Art convolutional neural network for autonomous navigation on nano-drones, to Tiny-PULP-Dronet, a novel methodology to squeeze by more than one order of magnitude model size. This massive reduction paves the way towards affordable multi-tasking on nano-drones, a fundamental requirement for achieving high-level intelligence.
  arXiv  Detail & Related papers  (2024-07-02T16:24:57Z)
• Gaussian Splatting to Real World Flight Navigation Transfer with Liquid Networks [93.38375271826202]
  We present a method to improve generalization and robustness to distribution shifts in sim-to-real visual quadrotor navigation tasks. We first build a simulator by integrating Gaussian splatting with quadrotor flight dynamics, and then, train robust navigation policies using Liquid neural networks. In this way, we obtain a full-stack imitation learning protocol that combines advances in 3D Gaussian splatting radiance field rendering, programming of expert demonstration training data, and the task understanding capabilities of Liquid networks.
  arXiv  Detail & Related papers  (2024-06-21T13:48:37Z)
• Object Detection in Thermal Images Using Deep Learning for Unmanned Aerial Vehicles [0.9208007322096533]
  This work presents a neural network model capable of recognizing small and tiny objects in thermal images collected by unmanned aerial vehicles. The backbone is developed based on the structure of YOLOv5 combined with the use of a transformer encoder at the end. The neck includes a BI-FPN block combined with the use of a sliding window and a transformer to increase the information fed into the prediction head.
  arXiv  Detail & Related papers  (2024-02-13T06:40:55Z)
• Human motion trajectory prediction using the Social Force Model for real-time and low computational cost applications [3.5970055082749655]
  We propose a novel trajectory prediction model, Social Force Generative Adversarial Network (SoFGAN) SoFGAN uses a Generative Adversarial Network (GAN) and Social Force Model (SFM) to generate different plausible people trajectories reducing collisions in a scene. We show that our method is more accurate in making predictions in UCY or BIWI datasets than most of the current state-of-the-art models and also reduces collisions in comparison to other approaches.
  arXiv  Detail & Related papers  (2023-11-17T15:32:21Z)
• UnLoc: A Universal Localization Method for Autonomous Vehicles using LiDAR, Radar and/or Camera Input [51.150605800173366]
  UnLoc is a novel unified neural modeling approach for localization with multi-sensor input in all weather conditions. Our method is extensively evaluated on Oxford Radar RobotCar, ApolloSouthBay and Perth-WA datasets.
  arXiv  Detail & Related papers  (2023-07-03T04:10:55Z)
• Resource-constrained FPGA Design for Satellite Component Feature Extraction [0.0]
  This work proposes use of neural network-based object detection algorithm that can be deployed on a resource-constrained FPGA. Hardware-in-the-loop experiments were performed on the ORION Maneuver Kinematics Simulator at Florida Tech. Results show the FPGA implementation increases the throughput and decreases latency while maintaining comparable accuracy.
  arXiv  Detail & Related papers  (2023-01-22T04:49:04Z)
• TransVisDrone: Spatio-Temporal Transformer for Vision-based Drone-to-Drone Detection in Aerial Videos [57.92385818430939]
  Drone-to-drone detection using visual feed has crucial applications, such as detecting drone collisions, detecting drone attacks, or coordinating flight with other drones. Existing methods are computationally costly, follow non-end-to-end optimization, and have complex multi-stage pipelines, making them less suitable for real-time deployment on edge devices. We propose a simple yet effective framework, itTransVisDrone, that provides an end-to-end solution with higher computational efficiency.
  arXiv  Detail & Related papers  (2022-10-16T03:05:13Z)
• Deep Learning for Real Time Satellite Pose Estimation on Low Power Edge TPU [58.720142291102135]
  In this paper we propose a pose estimation software exploiting neural network architectures. We show how low power machine learning accelerators could enable Artificial Intelligence exploitation in space.
  arXiv  Detail & Related papers  (2022-04-07T08:53:18Z)
• Edge Federated Learning Via Unit-Modulus Over-The-Air Computation (Extended Version) [64.76619508293966]
  This paper proposes a unit-modulus over-the-air computation (UM-AirComp) framework to facilitate efficient edge federated learning. It uploads simultaneously local model parameters and updates global model parameters via analog beamforming. We demonstrate the implementation of UM-AirComp in a vehicle-to-everything autonomous driving simulation platform.
  arXiv  Detail & Related papers  (2021-01-28T15:10:22Z)
• Vision-Based Autonomous Drone Control using Supervised Learning in Simulation [0.0]
  We propose a vision-based control approach using Supervised Learning for autonomous navigation and landing of MAVs in indoor environments. We trained a Convolutional Neural Network (CNN) that maps low resolution image and sensor input to high-level control commands. Our approach requires shorter training times than similar Reinforcement Learning approaches and can potentially overcome the limitations of manual data collection faced by comparable Supervised Learning approaches.
  arXiv  Detail & Related papers  (2020-09-09T13:45:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.