Lyapunov Neural ODE Feedback Control Policies

• URL: http://arxiv.org/abs/2409.00393v1
• Date: Sat, 31 Aug 2024 08:59:18 GMT
• Title: Lyapunov Neural ODE Feedback Control Policies
• Authors: Joshua Hang Sai Ip, Georgios Makrygiorgos, Ali Mesbah,
• Abstract summary: This paper presents a Lyapunov-NODE control (L-NODEC) approach to solving continuous-time optimal control problems. We establish that L-NODEC ensures exponential stability of the controlled system, as well as its adversarial robustness to uncertain initial conditions.
• Score: 6.165163123577486
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep neural networks are increasingly used as an effective way to represent control policies in a wide-range of learning-based control methods. For continuous-time optimal control problems (OCPs), which are central to many decision-making tasks, control policy learning can be cast as a neural ordinary differential equation (NODE) problem wherein state and control constraints are naturally accommodated. This paper presents a Lyapunov-NODE control (L-NODEC) approach to solving continuous-time OCPs for the case of stabilizing a known constrained nonlinear system around a terminal equilibrium point. We propose a Lyapunov loss formulation that incorporates a control-theoretic Lyapunov condition into the problem of learning a state-feedback neural control policy. We establish that L-NODEC ensures exponential stability of the controlled system, as well as its adversarial robustness to uncertain initial conditions. The performance of L-NODEC is illustrated on a benchmark double integrator problem and for optimal control of thermal dose delivery using a cold atmospheric plasma biomedical system. L-NODEC can substantially reduce the inference time necessary to reach the equilibrium state.

Related papers

• Lyapunov-stable Neural Control for State and Output Feedback: A Novel Formulation [67.63756749551924]
  Learning-based neural network (NN) control policies have shown impressive empirical performance in a wide range of tasks in robotics and control. Lyapunov stability guarantees over the region-of-attraction (ROA) for NN controllers with nonlinear dynamical systems are challenging to obtain. We demonstrate a new framework for learning NN controllers together with Lyapunov certificates using fast empirical falsification and strategic regularizations.
  arXiv  Detail & Related papers  (2024-04-11T17:49:15Z)
• A Neural RDE approach for continuous-time non-Markovian stochastic control problems [4.155942878350882]
  We propose a novel framework for continuous-time non-Markovian control problems by means of neural rough differential equations (Neural RDEs) Non-Markovianity naturally arises in control problems due to the time delay effects in the system coefficients or the driving noises. By modelling the control process as the solution of a Neural RDE driven by the state process, we show that the control-state joint dynamics are governed by an uncontrolled, augmented Neural RDE.
  arXiv  Detail & Related papers  (2023-06-25T14:30:33Z)
• Optimal control for state preparation in two-qubit open quantum systems driven by coherent and incoherent controls via GRAPE approach [77.34726150561087]
  We consider a model of two qubits driven by coherent and incoherent time-dependent controls. The dynamics of the system is governed by a Gorini-Kossakowski-Sudarshan-Lindblad master equation. We study evolution of the von Neumann entropy, purity, and one-qubit reduced density matrices under optimized controls.
  arXiv  Detail & Related papers  (2022-11-04T15:20:18Z)
• KCRL: Krasovskii-Constrained Reinforcement Learning with Guaranteed Stability in Nonlinear Dynamical Systems [66.9461097311667]
  We propose a model-based reinforcement learning framework with formal stability guarantees. The proposed method learns the system dynamics up to a confidence interval using feature representation. We show that KCRL is guaranteed to learn a stabilizing policy in a finite number of interactions with the underlying unknown system.
  arXiv  Detail & Related papers  (2022-06-03T17:27:04Z)
• Reliably-stabilizing piecewise-affine neural network controllers [5.203329540700177]
  A common problem affecting neural network (NN) approximations of model predictive control (MPC) policies is the lack of analytical tools to assess the stability of the closed-loop system under the action of the NN-based controller. We present a general procedure to quantify the performance of such a controller, or to design minimum complexity NNs that preserve the desirable properties of a given MPC scheme.
  arXiv  Detail & Related papers  (2021-11-13T20:01:43Z)
• A Theoretical Overview of Neural Contraction Metrics for Learning-based Control with Guaranteed Stability [7.963506386866862]
  This paper presents a neural network model of an optimal contraction metric and corresponding differential Lyapunov function. Its innovation lies in providing formal robustness guarantees for learning-based control frameworks.
  arXiv  Detail & Related papers  (2021-10-02T00:28:49Z)
• Stable Online Control of Linear Time-Varying Systems [49.41696101740271]
  COCO-LQ is an efficient online control algorithm that guarantees input-to-state stability for a large class of LTV systems. We empirically demonstrate the performance of COCO-LQ in both synthetic experiments and a power system frequency control example.
  arXiv  Detail & Related papers  (2021-04-29T06:18:49Z)
• Implicit energy regularization of neural ordinary-differential-equation control [3.5880535198436156]
  We present a versatile neural ordinary-differential-equation control (NODEC) framework with implicit energy regularization. We show that NODEC can steer dynamical systems towards a desired target state within a predefined amount of time.
  arXiv  Detail & Related papers  (2021-03-11T08:28:15Z)
• Lyapunov-Regularized Reinforcement Learning for Power System Transient Stability [5.634825161148484]
  This paper proposes a Lyapunov regularized RL approach for optimal frequency control for transient stability in lossy networks. Case study shows that introducing the Lyapunov regularization enables the controller to be stabilizing and achieve smaller losses.
  arXiv  Detail & Related papers  (2021-03-05T18:55:26Z)
• Gaussian Process-based Min-norm Stabilizing Controller for Control-Affine Systems with Uncertain Input Effects and Dynamics [90.81186513537777]
  We propose a novel compound kernel that captures the control-affine nature of the problem. We show that this resulting optimization problem is convex, and we call it Gaussian Process-based Control Lyapunov Function Second-Order Cone Program (GP-CLF-SOCP)
  arXiv  Detail & Related papers  (2020-11-14T01:27:32Z)
• Adaptive Control and Regret Minimization in Linear Quadratic Gaussian (LQG) Setting [91.43582419264763]
  We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty. LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model.
  arXiv  Detail & Related papers  (2020-03-12T19:56:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.