Kernel-Based Optimal Control: An Infinitesimal Generator Approach

• URL: http://arxiv.org/abs/2412.01591v2
• Date: Thu, 05 Dec 2024 10:22:00 GMT
• Title: Kernel-Based Optimal Control: An Infinitesimal Generator Approach
• Authors: Petar Bevanda, Nicolas Hoischen, Tobias Wittmann, Jan Brüdigam, Sandra Hirche, Boris Houska,
• Abstract summary: We present a novel approach for optimal control of nonlinear systems using infinitesimal generator learning within infinite-dimensional reproducing Hilbert kernel spaces. Our learning framework leverages data samples of system dynamics and stage cost functions, with only control penalties and constraints provided. We demonstrate that this approach seamlessly integrates with modern convex operator-theoretic Hamilton-Jacobi-Bellman recursions, enabling a data-driven solution to the optimal control problem.
• Score: 3.7411439050030393
• License:
• Abstract: This paper presents a novel approach for optimal control of nonlinear stochastic systems using infinitesimal generator learning within infinite-dimensional reproducing kernel Hilbert spaces. Our learning framework leverages data samples of system dynamics and stage cost functions, with only control penalties and constraints provided. The proposed method directly learns the diffusion operator of a controlled Fokker-Planck-Kolmogorov equation in an infinite-dimensional hypothesis space. This operator models the continuous-time evolution of the probability measure of the control system's state. We demonstrate that this approach seamlessly integrates with modern convex operator-theoretic Hamilton-Jacobi-Bellman recursions, enabling a data-driven solution to the optimal control problem. Furthermore, our statistical learning framework includes nonparametric estimators for uncontrolled forward infinitesimal generators as a special case. Numerical experiments, ranging from synthetic differential equations to simulated robotic systems, showcase the advantages of our approach compared to both modern data-driven and classical nonlinear programming methods for optimal control.

Related papers

• Learning Controlled Stochastic Differential Equations [61.82896036131116]
  This work proposes a novel method for estimating both drift and diffusion coefficients of continuous, multidimensional, nonlinear controlled differential equations with non-uniform diffusion. We provide strong theoretical guarantees, including finite-sample bounds for (L2), (Linfty), and risk metrics, with learning rates adaptive to coefficients' regularity. Our method is available as an open-source Python library.
  arXiv  Detail & Related papers  (2024-11-04T11:09:58Z)
• Data-Driven Optimal Feedback Laws via Kernel Mean Embeddings [3.007066256364399]
  We introduce kernel mean embeddings (KMEs) to identify the Markov transition operators associated with controlled diffusion processes. Unlike traditional dynamic programming methods, our approach exploits the Kernel trick'' to break the curse of dimensionality. We demonstrate the effectiveness of our method through numerical examples, highlighting its ability to solve a large class of nonlinear optimal control problems.
  arXiv  Detail & Related papers  (2024-07-23T11:53:03Z)
• Random Features Approximation for Control-Affine Systems [6.067043299145924]
  We propose two novel classes of nonlinear feature representations which capture control affine structure. Our methods make use of random features (RF) approximations, inheriting the expressiveness of kernel methods at a lower computational cost.
  arXiv  Detail & Related papers  (2024-06-10T17:54:57Z)
• Data-driven Nonlinear Model Reduction using Koopman Theory: Integrated Control Form and NMPC Case Study [56.283944756315066]
  We propose generic model structures combining delay-coordinate encoding of measurements and full-state decoding to integrate reduced Koopman modeling and state estimation. A case study demonstrates that our approach provides accurate control models and enables real-time capable nonlinear model predictive control of a high-purity cryogenic distillation column.
  arXiv  Detail & Related papers  (2024-01-09T11:54:54Z)
• Sample-efficient Model-based Reinforcement Learning for Quantum Control [0.2999888908665658]
  We propose a model-based reinforcement learning (RL) approach for noisy time-dependent gate optimization. We show an order of magnitude advantage in the sample complexity of our method over standard model-free RL. Our algorithm is well suited for controlling partially characterised one and two qubit systems.
  arXiv  Detail & Related papers  (2023-04-19T15:05:19Z)
• Physics-Informed Kernel Embeddings: Integrating Prior System Knowledge with Data-Driven Control [22.549914935697366]
  We present a method to incorporate priori knowledge into data-driven control algorithms using kernel embeddings. Our proposed approach incorporates prior knowledge of the system dynamics as a bias term in the kernel learning problem. We demonstrate the improved sample efficiency and out-of-sample generalization of our approach over a purely data-driven baseline.
  arXiv  Detail & Related papers  (2023-01-09T18:35:32Z)
• Deep Learning Approximation of Diffeomorphisms via Linear-Control Systems [91.3755431537592]
  We consider a control system of the form $dot x = sum_i=1lF_i(x)u_i$, with linear dependence in the controls. We use the corresponding flow to approximate the action of a diffeomorphism on a compact ensemble of points.
  arXiv  Detail & Related papers  (2021-10-24T08:57:46Z)
• Sparsity in Partially Controllable Linear Systems [56.142264865866636]
  We study partially controllable linear dynamical systems specified by an underlying sparsity pattern. Our results characterize those state variables which are irrelevant for optimal control.
  arXiv  Detail & Related papers  (2021-10-12T16:41:47Z)
• Gaussian Process-based Min-norm Stabilizing Controller for Control-Affine Systems with Uncertain Input Effects and Dynamics [90.81186513537777]
  We propose a novel compound kernel that captures the control-affine nature of the problem. We show that this resulting optimization problem is convex, and we call it Gaussian Process-based Control Lyapunov Function Second-Order Cone Program (GP-CLF-SOCP)
  arXiv  Detail & Related papers  (2020-11-14T01:27:32Z)
• Technical Report: Adaptive Control for Linearizable Systems Using On-Policy Reinforcement Learning [41.24484153212002]
  This paper proposes a framework for adaptively learning a feedback linearization-based tracking controller for an unknown system. It does not require the learned inverse model to be invertible at all instances of time. A simulated example of a double pendulum demonstrates the utility of the proposed theory.
  arXiv  Detail & Related papers  (2020-04-06T15:50:31Z)
• Adaptive Control and Regret Minimization in Linear Quadratic Gaussian (LQG) Setting [91.43582419264763]
  We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty. LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model.
  arXiv  Detail & Related papers  (2020-03-12T19:56:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.