Data-driven End-to-end Learning of Pole Placement Control for Nonlinear Dynamics via Koopman Invariant Subspaces

• URL: http://arxiv.org/abs/2208.08883v1
• Date: Tue, 16 Aug 2022 05:57:28 GMT
• Title: Data-driven End-to-end Learning of Pole Placement Control for Nonlinear Dynamics via Koopman Invariant Subspaces
• Authors: Tomoharu Iwata, Yoshinobu Kawahara
• Abstract summary: We propose a data-driven method for controlling black-box nonlinear dynamical systems based on the Koopman operator theory. A policy network is trained such that the eigenvalues of a Koopman operator of controlled dynamics are close to the target eigenvalues. We demonstrate that the proposed method achieves better performance than model-free reinforcement learning and model-based control with system identification.
• Score: 37.795752939016225
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a data-driven method for controlling the frequency and convergence rate of black-box nonlinear dynamical systems based on the Koopman operator theory. With the proposed method, a policy network is trained such that the eigenvalues of a Koopman operator of controlled dynamics are close to the target eigenvalues. The policy network consists of a neural network to find a Koopman invariant subspace, and a pole placement module to adjust the eigenvalues of the Koopman operator. Since the policy network is differentiable, we can train it in an end-to-end fashion using reinforcement learning. We demonstrate that the proposed method achieves better performance than model-free reinforcement learning and model-based control with system identification.

Related papers

• Non-Stationary Learning of Neural Networks with Automatic Soft Parameter Reset [98.52916361979503]
  We introduce a novel learning approach that automatically models and adapts to non-stationarity. We show empirically that our approach performs well in non-stationary supervised and off-policy reinforcement learning settings.
  arXiv  Detail & Related papers  (2024-11-06T16:32:40Z)
• Data-driven Nonlinear Model Reduction using Koopman Theory: Integrated Control Form and NMPC Case Study [56.283944756315066]
  We propose generic model structures combining delay-coordinate encoding of measurements and full-state decoding to integrate reduced Koopman modeling and state estimation. A case study demonstrates that our approach provides accurate control models and enables real-time capable nonlinear model predictive control of a high-purity cryogenic distillation column.
  arXiv  Detail & Related papers  (2024-01-09T11:54:54Z)
• Towards Data-driven LQR with KoopmanizingFlows [8.133902705930327]
  We propose a novel framework for learning linear time-invariant (LTI) models for a class of continuous-time non-autonomous nonlinear dynamics. We learn a finite representation of the Koopman operator that is linear in controls while concurrently learning meaningful lifting coordinates.
  arXiv  Detail & Related papers  (2022-01-27T17:02:03Z)
• Robust Value Iteration for Continuous Control Tasks [99.00362538261972]
  When transferring a control policy from simulation to a physical system, the policy needs to be robust to variations in the dynamics to perform well. We present Robust Fitted Value Iteration, which uses dynamic programming to compute the optimal value function on the compact state domain. We show that robust value is more robust compared to deep reinforcement learning algorithm and the non-robust version of the algorithm.
  arXiv  Detail & Related papers  (2021-05-25T19:48:35Z)
• Estimating Koopman operators for nonlinear dynamical systems: a nonparametric approach [77.77696851397539]
  The Koopman operator is a mathematical tool that allows for a linear description of non-linear systems. In this paper we capture their core essence as a dual version of the same framework, incorporating them into the Kernel framework. We establish a strong link between kernel methods and Koopman operators, leading to the estimation of the latter through Kernel functions.
  arXiv  Detail & Related papers  (2021-03-25T11:08:26Z)
• CKNet: A Convolutional Neural Network Based on Koopman Operator for Modeling Latent Dynamics from Pixels [5.286010070038216]
  We present a convolutional neural network (CNN) based on the Koopman operator (CKNet) to identify the latent dynamics from raw pixels. Experiments show that identified dynamics with 32-dim can predict validly 120 steps and generate clear images.
  arXiv  Detail & Related papers  (2021-02-19T23:29:08Z)
• Deep Learning of Koopman Representation for Control [0.0]
  The proposed approach relies on the Deep Neural Network based learning of Koopman operator for the purpose of control. The controller is purely data-driven and does not rely on a priori domain knowledge. The method is applied to two classic dynamical systems on OpenAI Gym environment to demonstrate the capability.
  arXiv  Detail & Related papers  (2020-10-15T06:41:24Z)
• Data Driven Control with Learned Dynamics: Model-Based versus Model-Free Approach [0.0]
  We compare two types of data-driven control methods, representing model-based and model-free approaches. One is a recently proposed method - Deep Koopman Representation for Control (DKRC), which utilizes a deep neural network to map an unknown nonlinear dynamical system to a high-dimensional linear system. The other is a classic model-free control method based on an actor-critic architecture - Deep Deterministic Policy Gradient (DDPG), which has been proved to be effective in various dynamical systems.
  arXiv  Detail & Related papers  (2020-06-16T22:18:21Z)
• Data-driven Koopman Operators for Model-based Shared Control of Human-Machine Systems [66.65503164312705]
  We present a data-driven shared control algorithm that can be used to improve a human operator's control of complex machines. Both the dynamics and information about the user's interaction are learned from observation through the use of a Koopman operator. We find that model-based shared control significantly improves task and control metrics when compared to a natural learning, or user only, control paradigm.
  arXiv  Detail & Related papers  (2020-06-12T14:14:07Z)
• Machine learning in spectral domain [4.724825031148412]
  tuning the eigenvalues correspond in fact to performing a global training of the neural network. spectral learning bound to the eigenvalues could be also employed for pre-training of deep neural networks.
  arXiv  Detail & Related papers  (2020-05-29T07:55:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.