Towards Data-driven LQR with KoopmanizingFlows

• URL: http://arxiv.org/abs/2201.11640v1
• Date: Thu, 27 Jan 2022 17:02:03 GMT
• Title: Towards Data-driven LQR with KoopmanizingFlows
• Authors: Petar Bevanda, Max Beier, Shahab Heshmati-Alamdari, Stefan Sosnowski, Sandra Hirche
• Abstract summary: We propose a novel framework for learning linear time-invariant (LTI) models for a class of continuous-time non-autonomous nonlinear dynamics. We learn a finite representation of the Koopman operator that is linear in controls while concurrently learning meaningful lifting coordinates.
• Score: 8.133902705930327
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a novel framework for learning linear time-invariant (LTI) models for a class of continuous-time non-autonomous nonlinear dynamics based on a representation of Koopman operators. In general, the operator is infinite-dimensional but, crucially, linear. To utilize it for efficient LTI control, we learn a finite representation of the Koopman operator that is linear in controls while concurrently learning meaningful lifting coordinates. For the latter, we rely on KoopmanizingFlows - a diffeomorphism-based representation of Koopman operators. With such a learned model, we can replace the nonlinear infinite-horizon optimal control problem with quadratic costs to that of a linear quadratic regulator (LQR), facilitating efficacious optimal control for nonlinear systems. The prediction and control efficacy of the proposed method is verified on simulation examples.

Related papers

• Uncertainty Modelling and Robust Observer Synthesis using the Koopman Operator [5.317624228510749]
  The Koopman operator allows nonlinear systems to be rewritten as infinite-dimensional linear systems. A finite-dimensional approximation of the Koopman operator can be identified directly from data. A population of several dozen motor drives is used to experimentally demonstrate the proposed method.
  arXiv  Detail & Related papers  (2024-10-01T20:31:18Z)
• Koopman-Assisted Reinforcement Learning [8.812992091278668]
  The Bellman equation and its continuous form, the Hamilton-Jacobi-Bellman (HJB) equation, are ubiquitous in reinforcement learning (RL) and control theory. This paper explores the connection between the data-driven Koopman operator and Decision Processes (MDPs) We develop two new RL algorithms to address these limitations.
  arXiv  Detail & Related papers  (2024-03-04T18:19:48Z)
• Data-driven Nonlinear Model Reduction using Koopman Theory: Integrated Control Form and NMPC Case Study [56.283944756315066]
  We propose generic model structures combining delay-coordinate encoding of measurements and full-state decoding to integrate reduced Koopman modeling and state estimation. A case study demonstrates that our approach provides accurate control models and enables real-time capable nonlinear model predictive control of a high-purity cryogenic distillation column.
  arXiv  Detail & Related papers  (2024-01-09T11:54:54Z)
• Deep Koopman Operator with Control for Nonlinear Systems [44.472875714432504]
  We propose an end-to-end deep learning framework to learn the Koopman embedding function and Koopman Operator. We first parameterize the embedding function and Koopman Operator with the neural network and train them end-to-end with the K-steps loss function. We then design an auxiliary control network to encode the nonlinear state-dependent control term to model the nonlinearity in control input.
  arXiv  Detail & Related papers  (2022-02-16T11:40:36Z)
• KoopmanizingFlows: Diffeomorphically Learning Stable Koopman Operators [7.447933533434023]
  We propose a novel framework for constructing linear time-invariant (LTI) models for a class of stable nonlinear dynamics. We learn the Koopman operator features without assuming a predefined library of functions or knowing the spectrum. We demonstrate the superior efficacy of the proposed method in comparison to a state-of-the-art method on the well-known LASA handwriting dataset.
  arXiv  Detail & Related papers  (2021-12-08T02:40:40Z)
• Estimating Koopman operators for nonlinear dynamical systems: a nonparametric approach [77.77696851397539]
  The Koopman operator is a mathematical tool that allows for a linear description of non-linear systems. In this paper we capture their core essence as a dual version of the same framework, incorporating them into the Kernel framework. We establish a strong link between kernel methods and Koopman operators, leading to the estimation of the latter through Kernel functions.
  arXiv  Detail & Related papers  (2021-03-25T11:08:26Z)
• LQF: Linear Quadratic Fine-Tuning [114.3840147070712]
  We present the first method for linearizing a pre-trained model that achieves comparable performance to non-linear fine-tuning. LQF consists of simple modifications to the architecture, loss function and optimization typically used for classification.
  arXiv  Detail & Related papers  (2020-12-21T06:40:20Z)
• Derivative-Based Koopman Operators for Real-Time Control of Robotic Systems [14.211417879279075]
  This paper presents a generalizable methodology for data-driven identification of nonlinear dynamics that bounds the model error. We construct a Koopman operator-based linear representation and utilize Taylor series accuracy analysis to derive an error bound. When combined with control, the Koopman representation of the nonlinear system has marginally better performance than competing nonlinear modeling methods.
  arXiv  Detail & Related papers  (2020-10-12T15:15:13Z)
• Learning the Linear Quadratic Regulator from Nonlinear Observations [135.66883119468707]
  We introduce a new problem setting for continuous control called the LQR with Rich Observations, or RichLQR. In our setting, the environment is summarized by a low-dimensional continuous latent state with linear dynamics and quadratic costs. Our results constitute the first provable sample complexity guarantee for continuous control with an unknown nonlinearity in the system model and general function approximation.
  arXiv  Detail & Related papers  (2020-10-08T07:02:47Z)
• Adaptive Control and Regret Minimization in Linear Quadratic Gaussian (LQG) Setting [91.43582419264763]
  We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty. LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model.
  arXiv  Detail & Related papers  (2020-03-12T19:56:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.