Reinforcement Learning with Formal Performance Metrics for Quadcopter Attitude Control under Non-nominal Contexts

• URL: http://arxiv.org/abs/2107.12942v1
• Date: Tue, 27 Jul 2021 16:58:19 GMT
• Title: Reinforcement Learning with Formal Performance Metrics for Quadcopter Attitude Control under Non-nominal Contexts
• Authors: Nicola Bernini, Mikhail Bessa, R\'emi Delmas, Arthur Gold, Eric Goubault, Romain Pennec, Sylvie Putot, Fran\c{c}ois Sillion
• Abstract summary: We develop a robust form of a signal temporal logic to quantitatively evaluate the vehicle's behavior and measure the performance of controllers. We discuss the robustness of the obtained controllers, both to partial loss of power for one rotor and to wind gusts and finish by drawing conclusions on practical controller design by reinforcement learning.
• Score: 2.198760145670348
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We explore the reinforcement learning approach to designing controllers by extensively discussing the case of a quadcopter attitude controller. We provide all details allowing to reproduce our approach, starting with a model of the dynamics of a crazyflie 2.0 under various nominal and non-nominal conditions, including partial motor failures and wind gusts. We develop a robust form of a signal temporal logic to quantitatively evaluate the vehicle's behavior and measure the performance of controllers. The paper thoroughly describes the choices in training algorithms, neural net architecture, hyperparameters, observation space in view of the different performance metrics we have introduced. We discuss the robustness of the obtained controllers, both to partial loss of power for one rotor and to wind gusts and finish by drawing conclusions on practical controller design by reinforcement learning.

Related papers

• Model-Free versus Model-Based Reinforcement Learning for Fixed-Wing UAV Attitude Control Under Varying Wind Conditions [1.474723404975345]
  This paper evaluates and compares the performance of model-free and model-based reinforcement learning for the attitude control of fixed-wing unmanned aerial vehicles using PID as a reference point. Results show that the Temporal Difference Model Predictive Control agent outperforms both the PID controller and other model-free reinforcement learning methods in terms of tracking accuracy and robustness.
  arXiv  Detail & Related papers  (2024-09-26T14:47:14Z)
• Modelling, Positioning, and Deep Reinforcement Learning Path Tracking Control of Scaled Robotic Vehicles: Design and Experimental Validation [3.807917169053206]
  Scaled robotic cars are commonly equipped with a hierarchical control acthiecture that includes tasks dedicated to vehicle state estimation and control. This paper covers both aspects by proposing (i) a federeted extended Kalman filter (FEKF) and (ii) a novel deep reinforcement learning (DRL) path tracking controller trained via an expert demonstrator. The experimentally validated model is used for (i) supporting the design of the FEKF and (ii) serving as a digital twin for training the proposed DRL-based path tracking algorithm.
  arXiv  Detail & Related papers  (2024-01-10T14:40:53Z)
• A Reinforcement Learning Approach for Robust Supervisory Control of UAVs Under Disturbances [1.8799681615947088]
  We present an approach to supervisory reinforcement learning control for unmanned aerial vehicles (UAVs) We formulate a supervisory control architecture that interleaves with extant embedded control and demonstrates robustness to environmental disturbances in the form of adverse wind conditions.
  arXiv  Detail & Related papers  (2023-05-21T19:00:06Z)
• Model-Based Reinforcement Learning with Isolated Imaginations [61.67183143982074]
  We propose Iso-Dream++, a model-based reinforcement learning approach. We perform policy optimization based on the decoupled latent imaginations. This enables long-horizon visuomotor control tasks to benefit from isolating mixed dynamics sources in the wild.
  arXiv  Detail & Related papers  (2023-03-27T02:55:56Z)
• Designing a Robust Low-Level Agnostic Controller for a Quadrotor with Actor-Critic Reinforcement Learning [0.38073142980732994]
  We introduce domain randomization during the training phase of a low-level waypoint guidance controller based on Soft Actor-Critic. We show that, by introducing a certain degree of uncertainty in quadrotor dynamics during training, we can obtain a controller that is capable to perform the proposed task using a larger variation of quadrotor parameters.
  arXiv  Detail & Related papers  (2022-10-06T14:58:19Z)
• Learning a Single Near-hover Position Controller for Vastly Different Quadcopters [56.37274861303324]
  This paper proposes an adaptive near-hover position controller for quadcopters. It can be deployed to quadcopters of very different mass, size and motor constants. It also shows rapid adaptation to unknown disturbances during runtime.
  arXiv  Detail & Related papers  (2022-09-19T17:55:05Z)
• Physics-Inspired Temporal Learning of Quadrotor Dynamics for Accurate Model Predictive Trajectory Tracking [76.27433308688592]
  Accurately modeling quadrotor's system dynamics is critical for guaranteeing agile, safe, and stable navigation. We present a novel Physics-Inspired Temporal Convolutional Network (PI-TCN) approach to learning quadrotor's system dynamics purely from robot experience. Our approach combines the expressive power of sparse temporal convolutions and dense feed-forward connections to make accurate system predictions.
  arXiv  Detail & Related papers  (2022-06-07T13:51:35Z)
• Nonprehensile Riemannian Motion Predictive Control [57.295751294224765]
  We introduce a novel Real-to-Sim reward analysis technique to reliably imagine and predict the outcome of taking possible actions for a real robotic platform. We produce a closed-loop controller to reactively push objects in a continuous action space. We observe that RMPC is robust in cluttered as well as occluded environments and outperforms the baselines.
  arXiv  Detail & Related papers  (2021-11-15T18:50:04Z)
• Is Disentanglement enough? On Latent Representations for Controllable Music Generation [78.8942067357231]
  In the absence of a strong generative decoder, disentanglement does not necessarily imply controllability. The structure of the latent space with respect to the VAE-decoder plays an important role in boosting the ability of a generative model to manipulate different attributes.
  arXiv  Detail & Related papers  (2021-08-01T18:37:43Z)
• Bidirectional Interaction between Visual and Motor Generative Models using Predictive Coding and Active Inference [68.8204255655161]
  We propose a neural architecture comprising a generative model for sensory prediction, and a distinct generative model for motor trajectories. We highlight how sequences of sensory predictions can act as rails guiding learning, control and online adaptation of motor trajectories.
  arXiv  Detail & Related papers  (2021-04-19T09:41:31Z)
• Robust Behavioral Cloning for Autonomous Vehicles using End-to-End Imitation Learning [0.4588028371034407]
  We present a lightweight pipeline for robust behavioral cloning of a human driver using end-to-end imitation learning. The proposed pipeline was employed to train and deploy three distinct driving behavior models onto a simulated vehicle.
  arXiv  Detail & Related papers  (2020-10-09T19:03:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.