Comparison of Model Predictive and Reinforcement Learning Methods for Fault Tolerant Control

• URL: http://arxiv.org/abs/2008.04403v1
• Date: Mon, 10 Aug 2020 20:22:15 GMT
• Title: Comparison of Model Predictive and Reinforcement Learning Methods for Fault Tolerant Control
• Authors: Ibrahim Ahmed, Hamed Khorasgani, Gautam Biswas
• Abstract summary: We present two adaptive fault-tolerant control schemes for a discrete time system based on hierarchical reinforcement learning. Experiments demonstrate that reinforcement learning-based controllers perform more robustly than model predictive controllers under faults, partially observable system models, and varying sensor noise levels.
• Score: 2.524528674141466
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A desirable property in fault-tolerant controllers is adaptability to system changes as they evolve during systems operations. An adaptive controller does not require optimal control policies to be enumerated for possible faults. Instead it can approximate one in real-time. We present two adaptive fault-tolerant control schemes for a discrete time system based on hierarchical reinforcement learning. We compare their performance against a model predictive controller in presence of sensor noise and persistent faults. The controllers are tested on a fuel tank model of a C-130 plane. Our experiments demonstrate that reinforcement learning-based controllers perform more robustly than model predictive controllers under faults, partially observable system models, and varying sensor noise levels.

Related papers

• Improving the Performance of Robust Control through Event-Triggered Learning [74.57758188038375]
  We propose an event-triggered learning algorithm that decides when to learn in the face of uncertainty in the LQR problem. We demonstrate improved performance over a robust controller baseline in a numerical example.
  arXiv  Detail & Related papers  (2022-07-28T17:36:37Z)
• Adaptive Model Predictive Control by Learning Classifiers [26.052368583196426]
  We propose an adaptive MPC variant that automatically estimates control and model parameters. We leverage recent results showing that BO can be formulated as a density ratio estimation. This is then integrated into a model predictive path integral control framework yielding robust controllers for a variety of challenging robotics tasks.
  arXiv  Detail & Related papers  (2022-03-13T23:22:12Z)
• Steady-State Error Compensation in Reference Tracking and Disturbance Rejection Problems for Reinforcement Learning-Based Control [0.9023847175654602]
  Reinforcement learning (RL) is a promising, upcoming topic in automatic control applications. Initiative action state augmentation (IASA) for actor-critic-based RL controllers is introduced. This augmentation does not require any expert knowledge, leaving the approach model free.
  arXiv  Detail & Related papers  (2022-01-31T16:29:19Z)
• Regret-optimal Estimation and Control [52.28457815067461]
  We show that the regret-optimal estimator and regret-optimal controller can be derived in state-space form. We propose regret-optimal analogs of Model-Predictive Control (MPC) and the Extended KalmanFilter (EKF) for systems with nonlinear dynamics.
  arXiv  Detail & Related papers  (2021-06-22T23:14:21Z)
• Learning-based vs Model-free Adaptive Control of a MAV under Wind Gust [0.2770822269241973]
  Navigation problems under unknown varying conditions are among the most important and well-studied problems in the control field. Recent model-free adaptive control methods aim at removing this dependency by learning the physical characteristics of the plant directly from sensor feedback. We propose a conceptually simple learning-based approach composed of a full state feedback controller, tuned robustly by a deep reinforcement learning framework.
  arXiv  Detail & Related papers  (2021-01-29T10:13:56Z)
• Safe and Efficient Model-free Adaptive Control via Bayesian Optimization [39.962395119933596]
  We propose a purely data-driven, model-free approach for adaptive control. tuning low-level controllers based solely on system data raises concerns on the underlying algorithm safety and computational performance. We numerically demonstrate for several types of disturbances that our approach is sample efficient, outperforms constrained Bayesian optimization in terms of safety, and achieves the performance optima computed by grid evaluation.
  arXiv  Detail & Related papers  (2021-01-19T19:15:00Z)
• Learning a Contact-Adaptive Controller for Robust, Efficient Legged Locomotion [95.1825179206694]
  We present a framework that synthesizes robust controllers for a quadruped robot. A high-level controller learns to choose from a set of primitives in response to changes in the environment. A low-level controller that utilizes an established control method to robustly execute the primitives.
  arXiv  Detail & Related papers  (2020-09-21T16:49:26Z)
• Anticipating the Long-Term Effect of Online Learning in Control [75.6527644813815]
  AntLer is a design algorithm for learning-based control laws that anticipates learning. We show that AntLer approximates an optimal solution arbitrarily accurately with probability one.
  arXiv  Detail & Related papers  (2020-07-24T07:00:14Z)
• Logarithmic Regret Bound in Partially Observable Linear Dynamical Systems [91.43582419264763]
  We study the problem of system identification and adaptive control in partially observable linear dynamical systems. We present the first model estimation method with finite-time guarantees in both open and closed-loop system identification. We show that AdaptOn is the first algorithm that achieves $textpolylogleft(Tright)$ regret in adaptive control of unknown partially observable linear dynamical systems.
  arXiv  Detail & Related papers  (2020-03-25T06:00:33Z)
• Adaptive Control and Regret Minimization in Linear Quadratic Gaussian (LQG) Setting [91.43582419264763]
  We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty. LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model.
  arXiv  Detail & Related papers  (2020-03-12T19:56:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.