Soft Actor-Critic Deep Reinforcement Learning for Fault Tolerant Flight Control

• URL: http://arxiv.org/abs/2202.09262v1
• Date: Wed, 16 Feb 2022 07:22:54 GMT
• Title: Soft Actor-Critic Deep Reinforcement Learning for Fault Tolerant Flight Control
• Abstract summary: offline trained cascaded Soft Actor-Critic Deep Reinforcement Learning controller is successful on highly coupled maneuvers. The controller is robust to six failure cases, including the rudder jammed at -15 deg, the aileron effectiveness reduced by 70%, a structural failure, icing and a backward c.g. shift.
• Score: 3.236217153362305
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Fault-tolerant flight control faces challenges, as developing a model-based controller for each unexpected failure is unrealistic, and online learning methods can handle limited system complexity due to their low sample efficiency. In this research, a model-free coupled-dynamics flight controller for a jet aircraft able to withstand multiple failure types is proposed. An offline trained cascaded Soft Actor-Critic Deep Reinforcement Learning controller is successful on highly coupled maneuvers, including a coordinated 40 degree bank climbing turn with a normalized Mean Absolute Error of 2.64%. The controller is robust to six failure cases, including the rudder jammed at -15 deg, the aileron effectiveness reduced by 70%, a structural failure, icing and a backward c.g. shift as the response is stable and the climbing turn is completed successfully. Robustness to biased sensor noise, atmospheric disturbances, and to varying initial flight conditions and reference signal shapes is also demonstrated.

Related papers

• A Machine Learning Framework for Fault Detection, Isolation, and Severity Prediction of Autonomous VTOL Aircraft [0.31498833540989407]
  Real-flight fault detection remains challenging due to sensor noise, environmental disturbances, and the nonlinear aerodynamics of multirotor platforms.<n>This study proposes a comprehensive machine-learning framework for fault detection, isolation, severity prediction using real flight data.
  arXiv  Detail & Related papers  (2026-09-12T23:03:05Z)
• Reinforcement Learning with Inner-loop Dynamics Estimator for Aerial Manipulation under Uncertainty [0.560928143828791]
  We present a hierarchical control framework that combines Reinforcement Learning (RL) with an inner-loop dynamics estimator to address this problem.<n>We validate the proposed approach on a custom quadrotor equipped with a 3-DoF manipulator through hardware experiments under varying payload conditions.
  arXiv  Detail & Related papers  (2026-06-15T12:16:01Z)
• Preserving Full 6-DOF Actuation Under Abrupt Total Rotor Failures: Passive Fault-Tolerant Flight Control Using a Biaxial-Tilt Hexacopter [21.18238702353788]
  Conventional multirotors suffer from a rapid collapse of attainable wrench space (AWS) under abrupt total rotor failures.<n>This paper addresses passive fault-tolerant flight of a biaxial-tilt over hexacopter (BTO) under abrupt total rotor failures.
  arXiv  Detail & Related papers  (2026-06-04T03:45:26Z)
• Aero-World: Action-Conditioned Aerial Video Generation from Inertial Controls [57.166020875486474]
  We propose textbfAero-World, a method for converting a pretrained image-to-video diffusion model into a controllable aerial video generator.<n>A frozen latent-space Physics Probe, trained independently on real video--IMU pairs, provides differentiable inertial-consistency supervision during LoRA finetuning.<n>On AeroBench, Aero-World improves mean AAS from 57.7 to 63.6 over action-only finetuning and gives a stronger quality-control trade-off than AirScape.
  arXiv  Detail & Related papers  (2026-05-19T12:02:17Z)
• Towards Robust LLM Post-Training: Automatic Failure Management for Reinforcement Fine-Tuning [44.75827047453428]
  We propose an automatic failure management framework for reinforcement fine-tuning.<n>RFT-FaultBench is the first benchmark for fine-grained failures in reinforcement fine-tuning.<n>RFT-FM shows strong capability in detecting, diagnosing, and mitigating RFT failures.
  arXiv  Detail & Related papers  (2026-05-06T02:50:00Z)
• Hypernetwork-Conditioned Reinforcement Learning for Robust Control of Fixed-Wing Aircraft under Actuator Failures [0.8164433158925594]
  This paper presents a reinforcement learning-based path-following controller for a small uncrewed aircraft system.<n>We consider parameter-efficient formulations based on Feature-wise Linear Modulation (FiLM) and Low-Rank Adaptation (LoRA)<n>We demonstrate that hypernetwork-conditioned policies can improve robustness compared to standard multilayer perceptron policies.
  arXiv  Detail & Related papers  (2026-04-03T18:50:31Z)
• A 26-Gram Butterfly-Inspired Robot Achieving Autonomous Tailless Flight [61.21669716392821]
  textitAirPulse is a 26-gram butterfly-inspired robot that achieves the first onboard, closed-loop controlled flight for a tailless two-winged platform at this scale.<n>It replicates key biomechanical traits of butterfly flight, utilizing low-aspect-ratio, compliant carbon-fiber-reinforced wings and low-frequency flapping that reproduces characteristic biological body undulations.
  arXiv  Detail & Related papers  (2026-02-06T15:58:50Z)
• Global End-Effector Pose Control of an Underactuated Aerial Manipulator via Reinforcement Learning [15.634803296068009]
  Aerial manipulators, which combine robotic arms with multi-rotor drones, face strict constraints on arm weight and mechanical complexity.<n>We study a lightweight 2-degree-of-freedom (DoF) arm mounted on a quadrotor via a differential mechanism, capable of full six-DoF end-effector pose control.
  arXiv  Detail & Related papers  (2025-12-24T10:00:01Z)
• Fault Tolerant Control of Mecanum Wheeled Mobile Robots [47.64303334083436]
  Mecanum wheeled mobile robots (MWMRs) are highly susceptible to actuator faults that degrade performance and risk mission failure.<n>Current fault tolerant control schemes for MWMRs target complete actuator failures like motor stall, ignoring partial faults e.g., in torque degradation.<n>We propose an FTC strategy handling both fault types, where we adopt posterior probability to learn real-time fault parameters.
  arXiv  Detail & Related papers  (2025-12-06T14:14:07Z)
• Explainable AI-Enhanced Supervisory Control for Robust Multi-Agent Robotic Systems [0.0]
  We present an explainable AI-enhanced supervisory control framework for multi-agent robotics.<n>We validated the approach in two contrasting domains, spacecraft formation flying and autonomous underwater vehicles.
  arXiv  Detail & Related papers  (2025-09-18T23:59:13Z)
• Why Change Your Controller When You Can Change Your Planner: Drag-Aware Trajectory Generation for Quadrotor Systems [10.101847906979435]
  Unmodeled aerodynamic drag forces from carried payloads can lead to catastrophic outcomes. We argue that adapting the trajectory generation component keeping the controller fixed can improve trajectory tracking. Our experiments in both simulation and on the Crazyflie hardware platform show that changing the planner reduces tracking error by as much as 83%.
  arXiv  Detail & Related papers  (2024-01-10T07:00:07Z)
• DATT: Deep Adaptive Trajectory Tracking for Quadrotor Control [62.24301794794304]
  Deep Adaptive Trajectory Tracking (DATT) is a learning-based approach that can precisely track arbitrary, potentially infeasible trajectories in the presence of large disturbances in the real world. DATT significantly outperforms competitive adaptive nonlinear and model predictive controllers for both feasible smooth and infeasible trajectories in unsteady wind fields. It can efficiently run online with an inference time less than 3.2 ms, less than 1/4 of the adaptive nonlinear model predictive control baseline.
  arXiv  Detail & Related papers  (2023-10-13T12:22:31Z)
• Actuator Trajectory Planning for UAVs with Overhead Manipulator using Reinforcement Learning [0.3222802562733786]
  We develop a UAV equipped with a controllable arm with two degrees of freedom to carry out actuation tasks on the fly. Our solution is based on employing a Q-learning method to control the trajectory of the tip of the arm, also called end-effector. Our method achieves 92% accuracy in terms of average displacement error using Q-learning with 15,000 episodes.
  arXiv  Detail & Related papers  (2023-08-24T15:06:23Z)
• Real-Time Model-Free Deep Reinforcement Learning for Force Control of a Series Elastic Actuator [56.11574814802912]
  State-of-the art robotic applications utilize series elastic actuators (SEAs) with closed-loop force control to achieve complex tasks such as walking, lifting, and manipulation. Model-free PID control methods are more prone to instability due to nonlinearities in the SEA. Deep reinforcement learning has proved to be an effective model-free method for continuous control tasks.
  arXiv  Detail & Related papers  (2023-04-11T00:51:47Z)
• A GOA-Based Fault-Tolerant Trajectory Tracking Control for an Underwater Vehicle of Multi-Thruster System without Actuator Saturation [9.371458775465825]
  This paper proposes an intelligent fault-tolerant control (FTC) strategy to tackle the trajectory tracking problem of an underwater vehicle (UV) under thruster damage (power loss) cases. In the proposed control strategy, the trajectory tracking component is formed by a refined backstepping algorithm that controls the velocity variation and a sliding mode control deducts the torque/force outputs.
  arXiv  Detail & Related papers  (2023-01-04T21:30:16Z)
• Online Model-Free Reinforcement Learning for the Automatic Control of a Flexible Wing Aircraft [2.3204178451683264]
  The control problem of the flexible wing aircraft is challenging due to the prevailing and high nonlinear deformations. An online control mechanism based on a value reinforcement learning process is developed for flexible wing aerial structures. It employs a model-free control policy framework and a guaranteed convergent adaptive learning architecture to solve the system's Bellman optimality equation.
  arXiv  Detail & Related papers  (2021-08-05T06:10:37Z)
• Regularizing Action Policies for Smooth Control with Reinforcement Learning [47.312768123967025]
  Conditioning for Action Policy Smoothness (CAPS) is an effective yet intuitive regularization on action policies. CAPS offers consistent improvement in the smoothness of the learned state-to-action mappings of neural network controllers. Tested on a real system, improvements in controller smoothness on a quadrotor drone resulted in an almost 80% reduction in power consumption.
  arXiv  Detail & Related papers  (2020-12-11T21:35:24Z)
• Learning a Contact-Adaptive Controller for Robust, Efficient Legged Locomotion [95.1825179206694]
  We present a framework that synthesizes robust controllers for a quadruped robot. A high-level controller learns to choose from a set of primitives in response to changes in the environment. A low-level controller that utilizes an established control method to robustly execute the primitives.
  arXiv  Detail & Related papers  (2020-09-21T16:49:26Z)
• Comparison of Model Predictive and Reinforcement Learning Methods for Fault Tolerant Control [2.524528674141466]
  We present two adaptive fault-tolerant control schemes for a discrete time system based on hierarchical reinforcement learning. Experiments demonstrate that reinforcement learning-based controllers perform more robustly than model predictive controllers under faults, partially observable system models, and varying sensor noise levels.
  arXiv  Detail & Related papers  (2020-08-10T20:22:15Z)
• Learning Compliance Adaptation in Contact-Rich Manipulation [81.40695846555955]
  We propose a novel approach for learning predictive models of force profiles required for contact-rich tasks. The approach combines an anomaly detection based on Bidirectional Gated Recurrent Units (Bi-GRU) and an adaptive force/impedance controller.
  arXiv  Detail & Related papers  (2020-05-01T05:23:34Z)
• Model-Based Meta-Reinforcement Learning for Flight with Suspended Payloads [69.21503033239985]
  Transporting suspended payloads is challenging for autonomous aerial vehicles. We propose a meta-learning approach that "learns how to learn" models of altered dynamics within seconds of post-connection flight data.
  arXiv  Detail & Related papers  (2020-04-23T17:43:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.