Neural Lyapunov Redesign

• URL: http://arxiv.org/abs/2006.03947v2
• Date: Mon, 23 Nov 2020 04:09:33 GMT
• Title: Neural Lyapunov Redesign
• Authors: Arash Mehrjou, Mohammad Ghavamzadeh, Bernhard Sch\"olkopf
• Abstract summary: Learning controllers must guarantee some notion of safety to ensure that it does not harm either the agent or the environment. Lyapunov functions are effective tools to assess stability in nonlinear dynamical systems. We propose a two-player collaborative algorithm that alternates between estimating a Lyapunov function and deriving a controller that gradually enlarges the stability region.
• Score: 36.2939747271983
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning controllers merely based on a performance metric has been proven effective in many physical and non-physical tasks in both control theory and reinforcement learning. However, in practice, the controller must guarantee some notion of safety to ensure that it does not harm either the agent or the environment. Stability is a crucial notion of safety, whose violation can certainly cause unsafe behaviors. Lyapunov functions are effective tools to assess stability in nonlinear dynamical systems. In this paper, we combine an improving Lyapunov function with automatic controller synthesis in an iterative fashion to obtain control policies with large safe regions. We propose a two-player collaborative algorithm that alternates between estimating a Lyapunov function and deriving a controller that gradually enlarges the stability region of the closed-loop system. We provide theoretical results on the class of systems that can be treated with the proposed algorithm and empirically evaluate the effectiveness of our method using an exemplary dynamical system.

Related papers

• Learning to Boost the Performance of Stable Nonlinear Systems [0.0]
  We tackle the performance-boosting problem with closed-loop stability guarantees. Our methods enable learning over arbitrarily deep neural network classes of performance-boosting controllers for stable nonlinear systems.
  arXiv  Detail & Related papers  (2024-05-01T21:11:29Z)
• Recursively Feasible Probabilistic Safe Online Learning with Control Barrier Functions [60.26921219698514]
  We introduce a model-uncertainty-aware reformulation of CBF-based safety-critical controllers. We then present the pointwise feasibility conditions of the resulting safety controller. We use these conditions to devise an event-triggered online data collection strategy.
  arXiv  Detail & Related papers  (2022-08-23T05:02:09Z)
• Sample-efficient Safe Learning for Online Nonlinear Control with Control Barrier Functions [35.9713619595494]
  Reinforcement Learning and continuous nonlinear control have been successfully deployed in multiple domains of complicated sequential decision-making tasks. Given the exploration nature of the learning process and the presence of model uncertainty, it is challenging to apply them to safety-critical control tasks. We propose a emphprovably efficient episodic safe learning framework for online control tasks.
  arXiv  Detail & Related papers  (2022-07-29T00:54:35Z)
• KCRL: Krasovskii-Constrained Reinforcement Learning with Guaranteed Stability in Nonlinear Dynamical Systems [66.9461097311667]
  We propose a model-based reinforcement learning framework with formal stability guarantees. The proposed method learns the system dynamics up to a confidence interval using feature representation. We show that KCRL is guaranteed to learn a stabilizing policy in a finite number of interactions with the underlying unknown system.
  arXiv  Detail & Related papers  (2022-06-03T17:27:04Z)
• Probabilistic robust linear quadratic regulators with Gaussian processes [73.0364959221845]
  Probabilistic models such as Gaussian processes (GPs) are powerful tools to learn unknown dynamical systems from data for subsequent use in control design. We present a novel controller synthesis for linearized GP dynamics that yields robust controllers with respect to a probabilistic stability margin.
  arXiv  Detail & Related papers  (2021-05-17T08:36:18Z)
• Closing the Closed-Loop Distribution Shift in Safe Imitation Learning [80.05727171757454]
  We treat safe optimization-based control strategies as experts in an imitation learning problem. We train a learned policy that can be cheaply evaluated at run-time and that provably satisfies the same safety guarantees as the expert.
  arXiv  Detail & Related papers  (2021-02-18T05:11:41Z)
• Enforcing robust control guarantees within neural network policies [76.00287474159973]
  We propose a generic nonlinear control policy class, parameterized by neural networks, that enforces the same provable robustness criteria as robust control. We demonstrate the power of this approach on several domains, improving in average-case performance over existing robust control methods and in worst-case stability over (non-robust) deep RL methods.
  arXiv  Detail & Related papers  (2020-11-16T17:14:59Z)
• Reinforcement Learning Control of Constrained Dynamic Systems with Uniformly Ultimate Boundedness Stability Guarantee [12.368097742148128]
  Reinforcement learning (RL) is promising for complicated nonlinear control problems. The data-based learning approach is notorious for not guaranteeing stability, which is the most fundamental property for any control system. In this paper, the classic Lyapunov's method is explored to analyze the uniformly ultimate boundedness stability (UUB) solely based on data.
  arXiv  Detail & Related papers  (2020-11-13T12:41:56Z)
• Actor-Critic Reinforcement Learning for Control with Stability Guarantee [9.400585561458712]
  Reinforcement Learning (RL) and its integration with deep learning have achieved impressive performance in various robotic control tasks. However, stability is not guaranteed in model-free RL by solely using data. We propose an actor-critic RL framework for control which can guarantee closed-loop stability by employing the classic Lyapunov's method in control theory.
  arXiv  Detail & Related papers  (2020-04-29T16:14:30Z)
• Neural Lyapunov Model Predictive Control: Learning Safe Global Controllers from Sub-optimal Examples [4.777323087050061]
  In many real-world and industrial applications, it is typical to have an existing control strategy, for instance, execution from a human operator. The objective of this work is to improve upon this unknown, safe but suboptimal policy by learning a new controller that retains safety and stability. The proposed algorithm alternatively learns the terminal cost and updates the MPC parameters according to a stability metric.
  arXiv  Detail & Related papers  (2020-02-21T16:57:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.