Robust Online Control with Model Misspecification
- URL: http://arxiv.org/abs/2107.07732v1
- Date: Fri, 16 Jul 2021 07:04:35 GMT
- Title: Robust Online Control with Model Misspecification
- Authors: Xinyi Chen, Udaya Ghai, Elad Hazan, Alexandre Megretski
- Abstract summary: We study online control of an unknown nonlinear dynamical system with model misspecification.
Our study focuses on robustness, which measures how much deviation from the assumed linear approximation can be tolerated.
- Score: 96.23493624553998
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We study online control of an unknown nonlinear dynamical system that is
approximated by a time-invariant linear system with model misspecification. Our
study focuses on robustness, which measures how much deviation from the assumed
linear approximation can be tolerated while maintaining a bounded $\ell_2$-gain
compared to the optimal control in hindsight. Some models cannot be stabilized
even with perfect knowledge of their coefficients: the robustness is limited by
the minimal distance between the assumed dynamics and the set of unstabilizable
dynamics. Therefore it is necessary to assume a lower bound on this distance.
Under this assumption, and with full observation of the $d$ dimensional state,
we describe an efficient controller that attains $\Omega(\frac{1}{\sqrt{d}})$
robustness together with an $\ell_2$-gain whose dimension dependence is near
optimal. We also give an inefficient algorithm that attains constant robustness
independent of the dimension, with a finite but sub-optimal $\ell_2$-gain.
Related papers
- Pseudonorm Approachability and Applications to Regret Minimization [73.54127663296906]
We convert high-dimensional $ell_infty$-approachability problems to low-dimensional pseudonorm approachability problems.
We develop an algorithmic theory of pseudonorm approachability, analogous to previous work on approachability for $ell$ and other norms.
arXiv Detail & Related papers (2023-02-03T03:19:14Z) - Efficient Online Linear Control with Stochastic Convex Costs and Unknown
Dynamics [0.0]
We present a computationally efficient algorithm that attains an optimal $sqrtT$ regret-rate against the best stabilizing linear controller.
In contrast to previous work, our algorithm is based on the Optimism in the Face of Uncertainty paradigm.
arXiv Detail & Related papers (2022-03-02T15:19:20Z) - Regret Bounds for Adaptive Nonlinear Control [14.489004143703825]
We prove the first finite-time regret bounds for adaptive nonlinear control with subject uncertainty in the setting.
We show that the regret suffered by certainty equivalence adaptive control, compared to an oracle controller with perfect knowledge of the unmodeled disturbances, is upper bounded by $widetildeO(sqrtT)$ in expectation.
arXiv Detail & Related papers (2020-11-26T03:01:09Z) - Information Theoretic Regret Bounds for Online Nonlinear Control [35.534829914047336]
We study the problem of sequential control in an unknown, nonlinear dynamical system.
This framework yields a general setting that permits discrete and continuous control inputs as well as non-smooth, non-differentiable dynamics.
We empirically show its application to a number of nonlinear control tasks and demonstrate the benefit of exploration for learning model dynamics.
arXiv Detail & Related papers (2020-06-22T17:46:48Z) - Learning Stabilizing Controllers for Unstable Linear Quadratic
Regulators from a Single Trajectory [85.29718245299341]
We study linear controllers under quadratic costs model also known as linear quadratic regulators (LQR)
We present two different semi-definite programs (SDP) which results in a controller that stabilizes all systems within an ellipsoid uncertainty set.
We propose an efficient data dependent algorithm -- textsceXploration -- that with high probability quickly identifies a stabilizing controller.
arXiv Detail & Related papers (2020-06-19T08:58:57Z) - Neural Contraction Metrics for Robust Estimation and Control: A Convex
Optimization Approach [6.646482960350819]
This paper presents a new deep learning-based framework for robust nonlinear estimation and control using the concept of a Neural Contraction Metric (NCM)
The NCM uses a deep long short-term memory recurrent neural network for a global approximation of an optimal contraction metric.
We demonstrate how to exploit NCMs to design an online optimal estimator and controller for nonlinear systems with bounded disturbances utilizing their duality.
arXiv Detail & Related papers (2020-06-08T05:29:38Z) - Sparse Identification of Nonlinear Dynamical Systems via Reweighted
$\ell_1$-regularized Least Squares [62.997667081978825]
This work proposes an iterative sparse-regularized regression method to recover governing equations of nonlinear systems from noisy state measurements.
The aim of this work is to improve the accuracy and robustness of the method in the presence of state measurement noise.
arXiv Detail & Related papers (2020-05-27T08:30:15Z) - Adaptive Control and Regret Minimization in Linear Quadratic Gaussian
(LQG) Setting [91.43582419264763]
We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty.
LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model.
arXiv Detail & Related papers (2020-03-12T19:56:38Z) - Naive Exploration is Optimal for Online LQR [49.681825576239355]
We show that the optimal regret scales as $widetildeTheta(sqrtd_mathbfu2 d_mathbfx T)$, where $T$ is the number of time steps, $d_mathbfu$ is the dimension of the input space, and $d_mathbfx$ is the dimension of the system state.
Our lower bounds rule out the possibility of a $mathrmpoly(logT)$-regret algorithm, which had been
arXiv Detail & Related papers (2020-01-27T03:44:54Z) - Optimistic robust linear quadratic dual control [4.94950858749529]
We present a dual control strategy that attempts to combine the performance of certainty equivalence, with the practical utility of robustness.
The formulation preserves structure in the representation of parametric uncertainty, which allows the controller to target reduction of uncertainty in the parameters that matter most' for the control task.
arXiv Detail & Related papers (2019-12-31T02:02:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.