Towards a Dimension-Free Understanding of Adaptive Linear Control
- URL: http://arxiv.org/abs/2103.10620v1
- Date: Fri, 19 Mar 2021 03:59:15 GMT
- Title: Towards a Dimension-Free Understanding of Adaptive Linear Control
- Authors: Juan C. Perdomo, Max Simchowitz, Alekh Agarwal, Peter Bartlett
- Abstract summary: We study the problem of adaptive control of the linear quadratic regulator for systems in very high, or even infinite dimension.
We provide the first regret bounds for LQR which hold for infinite dimensional systems.
- Score: 49.741419094419946
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We study the problem of adaptive control of the linear quadratic regulator
for systems in very high, or even infinite dimension. We demonstrate that while
sublinear regret requires finite dimensional inputs, the ambient state
dimension of the system need not be bounded in order to perform online control.
We provide the first regret bounds for LQR which hold for infinite dimensional
systems, replacing dependence on ambient dimension with more natural notions of
problem complexity. Our guarantees arise from a novel perturbation bound for
certainty equivalence which scales with the prediction error in estimating the
system parameters, without requiring consistent parameter recovery in more
stringent measures like the operator norm. When specialized to finite
dimensional settings, our bounds recover near optimal dimension and time
horizon dependence.
Related papers
- Stability Bounds for Learning-Based Adaptive Control of Discrete-Time
Multi-Dimensional Stochastic Linear Systems with Input Constraints [3.8004168340068336]
We consider the problem of adaptive stabilization for discrete-time, multi-dimensional systems with bounded control input constraints and unbounded disturbances.
We propose a certainty-equivalent control scheme which combines online parameter estimation with saturated linear control.
arXiv Detail & Related papers (2023-04-02T16:38:13Z) - Learning-Based Adaptive Control for Stochastic Linear Systems with Input
Constraints [3.8004168340068336]
We propose a certainty-equivalence scheme for adaptive control of scalar linear systems subject to additive, i.i.d.
Assuming that the system is at-worst marginally stable, mean square boundedness of the closed-loop system states is proven.
arXiv Detail & Related papers (2022-09-15T04:49:06Z) - Regret Lower Bounds for Learning Linear Quadratic Gaussian Systems [6.261682379939611]
We derive regret lower bounds exhibiting scaling on the order of magnitude $sqrtT$ in the time horizon $T$.
Our bounds accurately capture the role of control-theoretic parameters and we are able to show that systems that are hard to control are also hard to learn to control.
arXiv Detail & Related papers (2022-01-05T16:19:16Z) - Finite-time System Identification and Adaptive Control in Autoregressive
Exogenous Systems [79.67879934935661]
We study the problem of system identification and adaptive control of unknown ARX systems.
We provide finite-time learning guarantees for the ARX systems under both open-loop and closed-loop data collection.
arXiv Detail & Related papers (2021-08-26T18:00:00Z) - Robust Online Control with Model Misspecification [96.23493624553998]
We study online control of an unknown nonlinear dynamical system with model misspecification.
Our study focuses on robustness, which measures how much deviation from the assumed linear approximation can be tolerated.
arXiv Detail & Related papers (2021-07-16T07:04:35Z) - Dimension Free Generalization Bounds for Non Linear Metric Learning [61.193693608166114]
We provide uniform generalization bounds for two regimes -- the sparse regime, and a non-sparse regime.
We show that by relying on a different, new property of the solutions, it is still possible to provide dimension free generalization guarantees.
arXiv Detail & Related papers (2021-02-07T14:47:00Z) - Regret Bounds for Adaptive Nonlinear Control [14.489004143703825]
We prove the first finite-time regret bounds for adaptive nonlinear control with subject uncertainty in the setting.
We show that the regret suffered by certainty equivalence adaptive control, compared to an oracle controller with perfect knowledge of the unmodeled disturbances, is upper bounded by $widetildeO(sqrtT)$ in expectation.
arXiv Detail & Related papers (2020-11-26T03:01:09Z) - Adaptive Regret for Control of Time-Varying Dynamics [31.319502238224334]
We introduce the metric of it adaptive regret to the field of control.
Our main contribution is a novel efficient meta-algorithm.
The main technical innovation is the first adaptive regret bound for the more general framework of online convex optimization with memory.
arXiv Detail & Related papers (2020-07-08T19:40:34Z) - Adaptive Control and Regret Minimization in Linear Quadratic Gaussian
(LQG) Setting [91.43582419264763]
We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty.
LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model.
arXiv Detail & Related papers (2020-03-12T19:56:38Z) - Regret Minimization in Partially Observable Linear Quadratic Control [91.43582419264763]
We study the problem of regret in partially observable linear quadratic control systems when the model dynamics are unknown a priori.
We propose a novel way to decompose the regret and provide an end-to-end sublinear regret upper bound for partially observable linear quadratic control.
arXiv Detail & Related papers (2020-01-31T22:35:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.