A General Infrastructure and Workflow for Quadrotor Deep Reinforcement Learning and Reality Deployment

• URL: http://arxiv.org/abs/2504.15129v1
• Date: Mon, 21 Apr 2025 14:25:23 GMT
• Title: A General Infrastructure and Workflow for Quadrotor Deep Reinforcement Learning and Reality Deployment
• Authors: Kangyao Huang, Hao Wang, Yu Luo, Jingyu Chen, Jintao Chen, Xiangkui Zhang, Xiangyang Ji, Huaping Liu,
• Abstract summary: We propose a platform that enables seamless transfer of end-to-end deep reinforcement learning (DRL) policies to quadrotors.<n>Our platform provides rich types of environments including hovering, dynamic obstacle avoidance, trajectory tracking, balloon hitting, and planning in unknown environments.
• Score: 48.90852123901697
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deploying robot learning methods to a quadrotor in unstructured outdoor environments is an exciting task. Quadrotors operating in real-world environments by learning-based methods encounter several challenges: a large amount of simulator generated data required for training, strict demands for real-time processing onboard, and the sim-to-real gap caused by dynamic and noisy conditions. Current works have made a great breakthrough in applying learning-based methods to end-to-end control of quadrotors, but rarely mention the infrastructure system training from scratch and deploying to reality, which makes it difficult to reproduce methods and applications. To bridge this gap, we propose a platform that enables the seamless transfer of end-to-end deep reinforcement learning (DRL) policies. We integrate the training environment, flight dynamics control, DRL algorithms, the MAVROS middleware stack, and hardware into a comprehensive workflow and architecture that enables quadrotors' policies to be trained from scratch to real-world deployment in several minutes. Our platform provides rich types of environments including hovering, dynamic obstacle avoidance, trajectory tracking, balloon hitting, and planning in unknown environments, as a physical experiment benchmark. Through extensive empirical validation, we demonstrate the efficiency of proposed sim-to-real platform, and robust outdoor flight performance under real-world perturbations. Details can be found from our website https://emnavi.tech/AirGym/.

Related papers

• RAPID: Robust and Agile Planner Using Inverse Reinforcement Learning for Vision-Based Drone Navigation [9.25068777307471]
  This paper introduces a learning-based visual planner for agile drone flight in cluttered environments.<n>The proposed planner generates collision-free waypoints in milliseconds, enabling drones to perform agile maneuvers in complex environments without building separate perception, mapping, and planning modules.
  arXiv  Detail & Related papers  (2025-02-04T06:42:08Z)
• Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics [50.191655141020505]
  We introduce a novel framework for learning world models.<n>By providing a scalable and robust framework, we pave the way for adaptive and efficient robotic systems in real-world applications.
  arXiv  Detail & Related papers  (2025-01-17T10:39:09Z)
• Autonomous Vehicle Controllers From End-to-End Differentiable Simulation [60.05963742334746]
  We propose a differentiable simulator and design an analytic policy gradients (APG) approach to training AV controllers. Our proposed framework brings the differentiable simulator into an end-to-end training loop, where gradients of environment dynamics serve as a useful prior to help the agent learn a more grounded policy. We find significant improvements in performance and robustness to noise in the dynamics, as well as overall more intuitive human-like handling.
  arXiv  Detail & Related papers  (2024-09-12T11:50:06Z)
• Gaussian Splatting to Real World Flight Navigation Transfer with Liquid Networks [93.38375271826202]
  We present a method to improve generalization and robustness to distribution shifts in sim-to-real visual quadrotor navigation tasks. We first build a simulator by integrating Gaussian splatting with quadrotor flight dynamics, and then, train robust navigation policies using Liquid neural networks. In this way, we obtain a full-stack imitation learning protocol that combines advances in 3D Gaussian splatting radiance field rendering, programming of expert demonstration training data, and the task understanding capabilities of Liquid networks.
  arXiv  Detail & Related papers  (2024-06-21T13:48:37Z)
• Aquatic Navigation: A Challenging Benchmark for Deep Reinforcement Learning [53.3760591018817]
  We propose a new benchmarking environment for aquatic navigation using recent advances in the integration between game engines and Deep Reinforcement Learning. Specifically, we focus on PPO, one of the most widely accepted algorithms, and we propose advanced training techniques. Our empirical evaluation shows that a well-designed combination of these ingredients can achieve promising results.
  arXiv  Detail & Related papers  (2024-05-30T23:20:23Z)
• DRIFT: Deep Reinforcement Learning for Intelligent Floating Platforms Trajectories [18.420795137038677]
  Floating platforms serve as versatile test-beds to emulate micro-gravity environments on Earth. Our suite achieves robustness, adaptability, and good transferability from simulation to reality.
  arXiv  Detail & Related papers  (2023-10-06T14:11:35Z)
• Discrete Control in Real-World Driving Environments using Deep Reinforcement Learning [2.467408627377504]
  We introduce a framework (perception, planning, and control) in a real-world driving environment that transfers the real-world environments into gaming environments. We propose variations of existing Reinforcement Learning (RL) algorithms in a multi-agent setting to learn and execute the discrete control in real-world environments.
  arXiv  Detail & Related papers  (2022-11-29T04:24:03Z)
• Cloud-Edge Training Architecture for Sim-to-Real Deep Reinforcement Learning [0.8399688944263843]
  Deep reinforcement learning (DRL) is a promising approach to solve complex control tasks by learning policies through interactions with the environment. Sim-to-real approaches leverage simulations to pretrain DRL policies and then deploy them in the real world. This work proposes a distributed cloud-edge architecture to train DRL agents in the real world in real-time.
  arXiv  Detail & Related papers  (2022-03-04T10:27:01Z)
• Meta-Reinforcement Learning for Robotic Industrial Insertion Tasks [70.56451186797436]
  We study how to use meta-reinforcement learning to solve the bulk of the problem in simulation. We demonstrate our approach by training an agent to successfully perform challenging real-world insertion tasks.
  arXiv  Detail & Related papers  (2020-04-29T18:00:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.