LQGNet: Hybrid Model-Based and Data-Driven Linear Quadratic Stochastic Control

• URL: http://arxiv.org/abs/2210.12803v2
• Date: Tue, 25 Oct 2022 02:49:09 GMT
• Title: LQGNet: Hybrid Model-Based and Data-Driven Linear Quadratic Stochastic Control
• Authors: Solomon Goldgraber Casspi, Oliver Husser, Guy Revach, and Nir Shlezinger
• Abstract summary: quadratic control deals with finding an optimal control signal for a dynamical system in a setting with uncertainty. LQGNet is a controller that leverages data to operate under partially known dynamics. We show that LQGNet outperforms classic control by overcoming mismatched SS models.
• Score: 24.413595920205907
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Stochastic control deals with finding an optimal control signal for a dynamical system in a setting with uncertainty, playing a key role in numerous applications. The linear quadratic Gaussian (LQG) is a widely-used setting, where the system dynamics is represented as a linear Gaussian statespace (SS) model, and the objective function is quadratic. For this setting, the optimal controller is obtained in closed form by the separation principle. However, in practice, the underlying system dynamics often cannot be faithfully captured by a fully known linear Gaussian SS model, limiting its performance. Here, we present LQGNet, a stochastic controller that leverages data to operate under partially known dynamics. LQGNet augments the state tracking module of separation-based control with a dedicated trainable algorithm. The resulting system preserves the operation of classic LQG control while learning to cope with partially known SS models without having to fully identify the dynamics. We empirically show that LQGNet outperforms classic stochastic control by overcoming mismatched SS models.

Related papers

• Sample-efficient Model-based Reinforcement Learning for Quantum Control [0.2999888908665658]
  We propose a model-based reinforcement learning (RL) approach for noisy time-dependent gate optimization. We show an order of magnitude advantage in the sample complexity of our method over standard model-free RL. Our algorithm is well suited for controlling partially characterised one and two qubit systems.
  arXiv  Detail & Related papers  (2023-04-19T15:05:19Z)
• Bridging Model-based Safety and Model-free Reinforcement Learning through System Identification of Low Dimensional Linear Models [16.511440197186918]
  We propose a new method to combine model-based safety with model-free reinforcement learning. We show that a low-dimensional dynamical model is sufficient to capture the dynamics of the closed-loop system. We illustrate that the found linear model is able to provide guarantees by safety-critical optimal control framework.
  arXiv  Detail & Related papers  (2022-05-11T22:03:18Z)
• Sparsity in Partially Controllable Linear Systems [56.142264865866636]
  We study partially controllable linear dynamical systems specified by an underlying sparsity pattern. Our results characterize those state variables which are irrelevant for optimal control.
  arXiv  Detail & Related papers  (2021-10-12T16:41:47Z)
• Finite-time System Identification and Adaptive Control in Autoregressive Exogenous Systems [79.67879934935661]
  We study the problem of system identification and adaptive control of unknown ARX systems. We provide finite-time learning guarantees for the ARX systems under both open-loop and closed-loop data collection.
  arXiv  Detail & Related papers  (2021-08-26T18:00:00Z)
• KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics [84.18625250574853]
  We present KalmanNet, a real-time state estimator that learns from data to carry out Kalman filtering under non-linear dynamics. We numerically demonstrate that KalmanNet overcomes nonlinearities and model mismatch, outperforming classic filtering methods.
  arXiv  Detail & Related papers  (2021-07-21T12:26:46Z)
• Gaussian Process-based Min-norm Stabilizing Controller for Control-Affine Systems with Uncertain Input Effects and Dynamics [90.81186513537777]
  We propose a novel compound kernel that captures the control-affine nature of the problem. We show that this resulting optimization problem is convex, and we call it Gaussian Process-based Control Lyapunov Function Second-Order Cone Program (GP-CLF-SOCP)
  arXiv  Detail & Related papers  (2020-11-14T01:27:32Z)
• Information Theoretic Regret Bounds for Online Nonlinear Control [35.534829914047336]
  We study the problem of sequential control in an unknown, nonlinear dynamical system. This framework yields a general setting that permits discrete and continuous control inputs as well as non-smooth, non-differentiable dynamics. We empirically show its application to a number of nonlinear control tasks and demonstrate the benefit of exploration for learning model dynamics.
  arXiv  Detail & Related papers  (2020-06-22T17:46:48Z)
• Logarithmic Regret Bound in Partially Observable Linear Dynamical Systems [91.43582419264763]
  We study the problem of system identification and adaptive control in partially observable linear dynamical systems. We present the first model estimation method with finite-time guarantees in both open and closed-loop system identification. We show that AdaptOn is the first algorithm that achieves $textpolylogleft(Tright)$ regret in adaptive control of unknown partially observable linear dynamical systems.
  arXiv  Detail & Related papers  (2020-03-25T06:00:33Z)
• Adaptive Control and Regret Minimization in Linear Quadratic Gaussian (LQG) Setting [91.43582419264763]
  We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty. LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model.
  arXiv  Detail & Related papers  (2020-03-12T19:56:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.