Certainty Equivalent Quadratic Control for Markov Jump Systems

• URL: http://arxiv.org/abs/2105.12358v1
• Date: Wed, 26 May 2021 06:45:47 GMT
• Title: Certainty Equivalent Quadratic Control for Markov Jump Systems
• Authors: Zhe Du, Yahya Sattar, Davoud Ataee Tarzanagh, Laura Balzano, Samet Oymak and Necmiye Ozay
• Abstract summary: We investigate robustness aspects of certainty equivalent model-based optimal control for MJS with quadratic cost function. We provide explicit perturbation bounds which decay as $mathcalO(epsilon + eta)$ and $mathcalO((epsilon + eta)2)$ respectively.
• Score: 24.744481548320305
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Real-world control applications often involve complex dynamics subject to abrupt changes or variations. Markov jump linear systems (MJS) provide a rich framework for modeling such dynamics. Despite an extensive history, theoretical understanding of parameter sensitivities of MJS control is somewhat lacking. Motivated by this, we investigate robustness aspects of certainty equivalent model-based optimal control for MJS with quadratic cost function. Given the uncertainty in the system matrices and in the Markov transition matrix is bounded by $\epsilon$ and $\eta$ respectively, robustness results are established for (i) the solution to coupled Riccati equations and (ii) the optimal cost, by providing explicit perturbation bounds which decay as $\mathcal{O}(\epsilon + \eta)$ and $\mathcal{O}((\epsilon + \eta)^2)$ respectively.

Related papers

• Latent Linear Quadratic Regulator for Robotic Control Tasks [10.09131366605447]
  This paper presents a $textbfla$tent $textbfl$inear $textbfq$uadratic $textbfr$egulator (LaLQR) that maps the state space into a latent space. Experiments show LaLQR's superior efficiency and generalization compared to other baselines.
  arXiv  Detail & Related papers  (2024-07-15T15:22:52Z)
• Improving Sample Efficiency of Model-Free Algorithms for Zero-Sum Markov Games [66.2085181793014]
  We show that a model-free stage-based Q-learning algorithm can enjoy the same optimality in the $H$ dependence as model-based algorithms. Our algorithm features a key novel design of updating the reference value functions as the pair of optimistic and pessimistic value functions.
  arXiv  Detail & Related papers  (2023-08-17T08:34:58Z)
• Formal Controller Synthesis for Markov Jump Linear Systems with Uncertain Dynamics [64.72260320446158]
  We propose a method for synthesising controllers for Markov jump linear systems. Our method is based on a finite-state abstraction that captures both the discrete (mode-jumping) and continuous (stochastic linear) behaviour of the MJLS. We apply our method to multiple realistic benchmark problems, in particular, a temperature control and an aerial vehicle delivery problem.
  arXiv  Detail & Related papers  (2022-12-01T17:36:30Z)
• Gradient-Free Methods for Deterministic and Stochastic Nonsmooth Nonconvex Optimization [94.19177623349947]
  Non-smooth non optimization problems emerge in machine learning and business making. Two core challenges impede the development of efficient methods with finitetime convergence guarantee. Two-phase versions of GFM and SGFM are also proposed and proven to achieve improved large-deviation results.
  arXiv  Detail & Related papers  (2022-09-12T06:53:24Z)
• Identification and Adaptive Control of Markov Jump Systems: Sample Complexity and Regret Bounds [24.74448154832031]
  We consider the problem of controlling an unknown Markov jump linear system (MJS) to optimize a quadratic objective. We first provide a system identification algorithm for MJS to learn the dynamics in each mode. We then propose an adaptive control scheme that performs system identification together with certainty equivalent control.
  arXiv  Detail & Related papers  (2021-11-13T02:38:13Z)
• Finite-time System Identification and Adaptive Control in Autoregressive Exogenous Systems [79.67879934935661]
  We study the problem of system identification and adaptive control of unknown ARX systems. We provide finite-time learning guarantees for the ARX systems under both open-loop and closed-loop data collection.
  arXiv  Detail & Related papers  (2021-08-26T18:00:00Z)
• Robust Online Control with Model Misspecification [96.23493624553998]
  We study online control of an unknown nonlinear dynamical system with model misspecification. Our study focuses on robustness, which measures how much deviation from the assumed linear approximation can be tolerated.
  arXiv  Detail & Related papers  (2021-07-16T07:04:35Z)
• A conditional one-output likelihood formulation for multitask Gaussian processes [0.0]
  Multitask Gaussian processes (MTGP) are the Gaussian process framework's solution for multioutput regression problems. Here we introduce a novel approach that simplifies the multitask learning. We show that it is computationally competitive with state of the art options.
  arXiv  Detail & Related papers  (2020-06-05T14:59:06Z)
• Adaptive Control and Regret Minimization in Linear Quadratic Gaussian (LQG) Setting [91.43582419264763]
  We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty. LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model.
  arXiv  Detail & Related papers  (2020-03-12T19:56:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.