Developmental Reinforcement Learning of Control Policy of a Quadcopter UAV with Thrust Vectoring Rotors

• URL: http://arxiv.org/abs/2007.07793v1
• Date: Wed, 15 Jul 2020 16:17:29 GMT
• Title: Developmental Reinforcement Learning of Control Policy of a Quadcopter UAV with Thrust Vectoring Rotors
• Authors: Aditya M. Deshpande and Rumit Kumar and Ali A. Minai and Manish Kumar
• Abstract summary: We present a novel developmental reinforcement learning-based controller for a quadcopter with thrust vectoring capabilities. The control policy of this robot is learned using the policy transfer from the learned controller of the quadcopter. The performance of the learned policy is evaluated by physics-based simulations for the tasks of hovering and way-point navigation.
• Score: 1.0057838324294686
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we present a novel developmental reinforcement learning-based controller for a quadcopter with thrust vectoring capabilities. This multirotor UAV design has tilt-enabled rotors. It utilizes the rotor force magnitude and direction to achieve the desired state during flight. The control policy of this robot is learned using the policy transfer from the learned controller of the quadcopter (comparatively simple UAV design without thrust vectoring). This approach allows learning a control policy for systems with multiple inputs and multiple outputs. The performance of the learned policy is evaluated by physics-based simulations for the tasks of hovering and way-point navigation. The flight simulations utilize a flight controller based on reinforcement learning without any additional PID components. The results show faster learning with the presented approach as opposed to learning the control policy from scratch for this new UAV design created by modifications in a conventional quadcopter, i.e., the addition of more degrees of freedom (4-actuators in conventional quadcopter to 8-actuators in tilt-rotor quadcopter). We demonstrate the robustness of our learned policy by showing the recovery of the tilt-rotor platform in the simulation from various non-static initial conditions in order to reach a desired state. The developmental policy for the tilt-rotor UAV also showed superior fault tolerance when compared with the policy learned from the scratch. The results show the ability of the presented approach to bootstrap the learned behavior from a simpler system (lower-dimensional action-space) to a more complex robot (comparatively higher-dimensional action-space) and reach better performance faster.

Related papers

• Autonomous Vehicle Controllers From End-to-End Differentiable Simulation [60.05963742334746]
  We propose a differentiable simulator and design an analytic policy gradients (APG) approach to training AV controllers. Our proposed framework brings the differentiable simulator into an end-to-end training loop, where gradients of environment dynamics serve as a useful prior to help the agent learn a more grounded policy. We find significant improvements in performance and robustness to noise in the dynamics, as well as overall more intuitive human-like handling.
  arXiv  Detail & Related papers  (2024-09-12T11:50:06Z)
• From Flies to Robots: Inverted Landing in Small Quadcopters with Dynamic Perching [15.57055572401334]
  Inverted landing is a routine behavior among a number of animal fliers. We develop a control policy general to arbitrary ceiling-approach conditions. We successfully achieved a range of robust inverted-landing behaviors in small quadcopters.
  arXiv  Detail & Related papers  (2024-02-29T21:09:08Z)
• Improving Wind Resistance Performance of Cascaded PID Controlled Quadcopters using Residual Reinforcement Learning [0.0]
  Wind resistance control is an essential feature for quadcopters to maintain their position to avoid deviation from target position. We propose a residual reinforcement learning based approach to build a wind resistance controller of a quadcopter. Our controller reduces the position deviation by approximately 50% compared to the quadcopter controlled with the conventional cascaded PID controller.
  arXiv  Detail & Related papers  (2023-08-03T09:29:19Z)
• Inverted Landing in a Small Aerial Robot via Deep Reinforcement Learning for Triggering and Control of Rotational Maneuvers [11.29285364660789]
  Inverted landing in a rapid and robust manner is a challenging feat for aerial robots, especially while depending entirely on onboard sensing and computation. Previous work has identified a direct causal connection between a series of onboard visual cues and kinematic actions that allow for reliable execution of this challenging aerobatic maneuver in small aerial robots. In this work, we first utilized Deep Reinforcement Learning and a physics-based simulation to obtain a general, optimal control policy for robust inverted landing.
  arXiv  Detail & Related papers  (2022-09-22T14:38:10Z)
• Learning a Single Near-hover Position Controller for Vastly Different Quadcopters [56.37274861303324]
  This paper proposes an adaptive near-hover position controller for quadcopters. It can be deployed to quadcopters of very different mass, size and motor constants. It also shows rapid adaptation to unknown disturbances during runtime.
  arXiv  Detail & Related papers  (2022-09-19T17:55:05Z)
• Adapting Rapid Motor Adaptation for Bipedal Robots [73.5914982741483]
  We leverage recent advances in rapid adaptation for locomotion control, and extend them to work on bipedal robots. A-RMA adapts the base policy for the imperfect extrinsics estimator by finetuning it using model-free RL. We demonstrate that A-RMA outperforms a number of RL-based baseline controllers and model-based controllers in simulation.
  arXiv  Detail & Related papers  (2022-05-30T17:59:09Z)
• Neural-Fly Enables Rapid Learning for Agile Flight in Strong Winds [96.74836678572582]
  We present a learning-based approach that allows rapid online adaptation by incorporating pretrained representations through deep learning. Neural-Fly achieves precise flight control with substantially smaller tracking error than state-of-the-art nonlinear and adaptive controllers.
  arXiv  Detail & Related papers  (2022-05-13T21:55:28Z)
• Reinforcement Learning with Formal Performance Metrics for Quadcopter Attitude Control under Non-nominal Contexts [2.198760145670348]
  We develop a robust form of a signal temporal logic to quantitatively evaluate the vehicle's behavior and measure the performance of controllers. We discuss the robustness of the obtained controllers, both to partial loss of power for one rotor and to wind gusts and finish by drawing conclusions on practical controller design by reinforcement learning.
  arXiv  Detail & Related papers  (2021-07-27T16:58:19Z)
• Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots [121.42930679076574]
  We present a model-free reinforcement learning framework for training robust locomotion policies in simulation. domain randomization is used to encourage the policies to learn behaviors that are robust across variations in system dynamics. We demonstrate this on versatile walking behaviors such as tracking a target walking velocity, walking height, and turning yaw.
  arXiv  Detail & Related papers  (2021-03-26T07:14:01Z)
• Learning a Contact-Adaptive Controller for Robust, Efficient Legged Locomotion [95.1825179206694]
  We present a framework that synthesizes robust controllers for a quadruped robot. A high-level controller learns to choose from a set of primitives in response to changes in the environment. A low-level controller that utilizes an established control method to robustly execute the primitives.
  arXiv  Detail & Related papers  (2020-09-21T16:49:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.