Regret Bounds for Adaptive Nonlinear Control
- URL: http://arxiv.org/abs/2011.13101v1
- Date: Thu, 26 Nov 2020 03:01:09 GMT
- Title: Regret Bounds for Adaptive Nonlinear Control
- Authors: Nicholas M. Boffi and Stephen Tu and Jean-Jacques E. Slotine
- Abstract summary: We prove the first finite-time regret bounds for adaptive nonlinear control with subject uncertainty in the setting.
We show that the regret suffered by certainty equivalence adaptive control, compared to an oracle controller with perfect knowledge of the unmodeled disturbances, is upper bounded by $widetildeO(sqrtT)$ in expectation.
- Score: 14.489004143703825
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We study the problem of adaptively controlling a known discrete-time
nonlinear system subject to unmodeled disturbances. We prove the first
finite-time regret bounds for adaptive nonlinear control with matched
uncertainty in the stochastic setting, showing that the regret suffered by
certainty equivalence adaptive control, compared to an oracle controller with
perfect knowledge of the unmodeled disturbances, is upper bounded by
$\widetilde{O}(\sqrt{T})$ in expectation. Furthermore, we show that when the
input is subject to a $k$ timestep delay, the regret degrades to
$\widetilde{O}(k \sqrt{T})$. Our analysis draws connections between classical
stability notions in nonlinear control theory (Lyapunov stability and
contraction theory) and modern regret analysis from online convex optimization.
The use of stability theory allows us to analyze the challenging
infinite-horizon single trajectory setting.
Related papers
- Stability Bounds for Learning-Based Adaptive Control of Discrete-Time
Multi-Dimensional Stochastic Linear Systems with Input Constraints [3.8004168340068336]
We consider the problem of adaptive stabilization for discrete-time, multi-dimensional systems with bounded control input constraints and unbounded disturbances.
We propose a certainty-equivalent control scheme which combines online parameter estimation with saturated linear control.
arXiv Detail & Related papers (2023-04-02T16:38:13Z) - Learning-Based Adaptive Control for Stochastic Linear Systems with Input
Constraints [3.8004168340068336]
We propose a certainty-equivalence scheme for adaptive control of scalar linear systems subject to additive, i.i.d.
Assuming that the system is at-worst marginally stable, mean square boundedness of the closed-loop system states is proven.
arXiv Detail & Related papers (2022-09-15T04:49:06Z) - Finite-time System Identification and Adaptive Control in Autoregressive
Exogenous Systems [79.67879934935661]
We study the problem of system identification and adaptive control of unknown ARX systems.
We provide finite-time learning guarantees for the ARX systems under both open-loop and closed-loop data collection.
arXiv Detail & Related papers (2021-08-26T18:00:00Z) - Robust Online Control with Model Misspecification [96.23493624553998]
We study online control of an unknown nonlinear dynamical system with model misspecification.
Our study focuses on robustness, which measures how much deviation from the assumed linear approximation can be tolerated.
arXiv Detail & Related papers (2021-07-16T07:04:35Z) - Reinforcement learning for linear-convex models with jumps via stability
analysis of feedback controls [7.969435896173812]
We study a finite linear-time continuous-time horizon learning problems an episodic setting.
In this problem, the unknown jump-dif process is controlled to nonsmooth convex costs.
arXiv Detail & Related papers (2021-04-19T13:50:52Z) - Towards a Dimension-Free Understanding of Adaptive Linear Control [49.741419094419946]
We study the problem of adaptive control of the linear quadratic regulator for systems in very high, or even infinite dimension.
We provide the first regret bounds for LQR which hold for infinite dimensional systems.
arXiv Detail & Related papers (2021-03-19T03:59:15Z) - Adaptive Control and Regret Minimization in Linear Quadratic Gaussian
(LQG) Setting [91.43582419264763]
We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty.
LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model.
arXiv Detail & Related papers (2020-03-12T19:56:38Z) - Regret Minimization in Partially Observable Linear Quadratic Control [91.43582419264763]
We study the problem of regret in partially observable linear quadratic control systems when the model dynamics are unknown a priori.
We propose a novel way to decompose the regret and provide an end-to-end sublinear regret upper bound for partially observable linear quadratic control.
arXiv Detail & Related papers (2020-01-31T22:35:08Z) - Improper Learning for Non-Stochastic Control [78.65807250350755]
We consider the problem of controlling a possibly unknown linear dynamical system with adversarial perturbations, adversarially chosen convex loss functions, and partially observed states.
Applying online descent to this parametrization yields a new controller which attains sublinear regret vs. a large class of closed-loop policies.
Our bounds are the first in the non-stochastic control setting that compete with emphall stabilizing linear dynamical controllers.
arXiv Detail & Related papers (2020-01-25T02:12:48Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.