Neural Lyapunov Model Predictive Control: Learning Safe Global Controllers from Sub-optimal Examples

• URL: http://arxiv.org/abs/2002.10451v2
• Date: Thu, 3 Jun 2021 14:37:05 GMT
• Title: Neural Lyapunov Model Predictive Control: Learning Safe Global Controllers from Sub-optimal Examples
• Authors: Mayank Mittal, Marco Gallieri, Alessio Quaglino, Seyed Sina Mirrazavi Salehian, Jan Koutn\'ik
• Abstract summary: In many real-world and industrial applications, it is typical to have an existing control strategy, for instance, execution from a human operator. The objective of this work is to improve upon this unknown, safe but suboptimal policy by learning a new controller that retains safety and stability. The proposed algorithm alternatively learns the terminal cost and updates the MPC parameters according to a stability metric.
• Score: 4.777323087050061
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With a growing interest in data-driven control techniques, Model Predictive Control (MPC) provides an opportunity to exploit the surplus of data reliably, particularly while taking safety and stability into account. In many real-world and industrial applications, it is typical to have an existing control strategy, for instance, execution from a human operator. The objective of this work is to improve upon this unknown, safe but suboptimal policy by learning a new controller that retains safety and stability. Learning how to be safe is achieved directly from data and from a knowledge of the system constraints. The proposed algorithm alternatively learns the terminal cost and updates the MPC parameters according to a stability metric. The terminal cost is constructed as a Lyapunov function neural network with the aim of recovering or extending the stable region of the initial demonstrator using a short prediction horizon. Theorems that characterize the stability and performance of the learned MPC in the bearing of model uncertainties and sub-optimality due to function approximation are presented. The efficacy of the proposed algorithm is demonstrated on non-linear continuous control tasks with soft constraints. The proposed approach can improve upon the initial demonstrator also in practice and achieve better stability than popular reinforcement learning baselines.

Related papers

• Safe and Stable Closed-Loop Learning for Neural-Network-Supported Model Predictive Control [0.0]
  We consider safe learning of parametrized predictive controllers that operate with incomplete information about the underlying process. Our method focuses on the system's overall long-term performance in closed-loop while keeping it safe and stable. We explicitly incorporated stability information in the Bayesian-optimization-based learning procedure, thereby achieving rigorous probabilistic safety guarantees.
  arXiv  Detail & Related papers  (2024-09-16T11:03:58Z)
• Learning to Boost the Performance of Stable Nonlinear Systems [0.0]
  We tackle the performance-boosting problem with closed-loop stability guarantees. Our methods enable learning over arbitrarily deep neural network classes of performance-boosting controllers for stable nonlinear systems.
  arXiv  Detail & Related papers  (2024-05-01T21:11:29Z)
• Meta-Learning Priors for Safe Bayesian Optimization [72.8349503901712]
  We build on a meta-learning algorithm, F-PACOH, capable of providing reliable uncertainty quantification in settings of data scarcity. As core contribution, we develop a novel framework for choosing safety-compliant priors in a data-riven manner. On benchmark functions and a high-precision motion system, we demonstrate that our meta-learned priors accelerate the convergence of safe BO approaches.
  arXiv  Detail & Related papers  (2022-10-03T08:38:38Z)
• Recursively Feasible Probabilistic Safe Online Learning with Control Barrier Functions [60.26921219698514]
  We introduce a model-uncertainty-aware reformulation of CBF-based safety-critical controllers. We then present the pointwise feasibility conditions of the resulting safety controller. We use these conditions to devise an event-triggered online data collection strategy.
  arXiv  Detail & Related papers  (2022-08-23T05:02:09Z)
• Pointwise Feasibility of Gaussian Process-based Safety-Critical Control under Model Uncertainty [77.18483084440182]
  Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs) are popular tools for enforcing safety and stability of a controlled system, respectively. We present a Gaussian Process (GP)-based approach to tackle the problem of model uncertainty in safety-critical controllers that use CBFs and CLFs.
  arXiv  Detail & Related papers  (2021-06-13T23:08:49Z)
• Probabilistic robust linear quadratic regulators with Gaussian processes [73.0364959221845]
  Probabilistic models such as Gaussian processes (GPs) are powerful tools to learn unknown dynamical systems from data for subsequent use in control design. We present a novel controller synthesis for linearized GP dynamics that yields robust controllers with respect to a probabilistic stability margin.
  arXiv  Detail & Related papers  (2021-05-17T08:36:18Z)
• Closing the Closed-Loop Distribution Shift in Safe Imitation Learning [80.05727171757454]
  We treat safe optimization-based control strategies as experts in an imitation learning problem. We train a learned policy that can be cheaply evaluated at run-time and that provably satisfies the same safety guarantees as the expert.
  arXiv  Detail & Related papers  (2021-02-18T05:11:41Z)
• Reinforcement Learning Control of Constrained Dynamic Systems with Uniformly Ultimate Boundedness Stability Guarantee [12.368097742148128]
  Reinforcement learning (RL) is promising for complicated nonlinear control problems. The data-based learning approach is notorious for not guaranteeing stability, which is the most fundamental property for any control system. In this paper, the classic Lyapunov's method is explored to analyze the uniformly ultimate boundedness stability (UUB) solely based on data.
  arXiv  Detail & Related papers  (2020-11-13T12:41:56Z)
• Chance-Constrained Trajectory Optimization for Safe Exploration and Learning of Nonlinear Systems [81.7983463275447]
  Learning-based control algorithms require data collection with abundant supervision for training. We present a new approach for optimal motion planning with safe exploration that integrates chance-constrained optimal control with dynamics learning and feedback control.
  arXiv  Detail & Related papers  (2020-05-09T05:57:43Z)
• Actor-Critic Reinforcement Learning for Control with Stability Guarantee [9.400585561458712]
  Reinforcement Learning (RL) and its integration with deep learning have achieved impressive performance in various robotic control tasks. However, stability is not guaranteed in model-free RL by solely using data. We propose an actor-critic RL framework for control which can guarantee closed-loop stability by employing the classic Lyapunov's method in control theory.
  arXiv  Detail & Related papers  (2020-04-29T16:14:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.