Suboptimality analysis of receding horizon quadratic control with unknown linear systems and its applications in learning-based control
- URL: http://arxiv.org/abs/2301.07876v2
- Date: Tue, 9 Apr 2024 03:11:26 GMT
- Title: Suboptimality analysis of receding horizon quadratic control with unknown linear systems and its applications in learning-based control
- Authors: Shengling Shi, Anastasios Tsiamis, Bart De Schutter,
- Abstract summary: We analyze how the trade-off between the modeling error, the terminal value function error, and the prediction horizon affects the performance of a nominal receding-horizon linear quadratic (LQ) controller.
We show that when an infinite horizon is desired, a finite prediction horizon that is larger than the controllability index can be sufficient for achieving a near-optimal performance.
- Score: 14.279848166377668
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In this work, we aim to analyze how the trade-off between the modeling error, the terminal value function error, and the prediction horizon affects the performance of a nominal receding-horizon linear quadratic (LQ) controller. By developing a novel perturbation result of the Riccati difference equation, a novel performance upper bound is obtained and suggests that for many cases, the prediction horizon can be either one or infinity to improve the control performance, depending on the relative difference between the modeling error and the terminal value function error. The result also shows that when an infinite horizon is desired, a finite prediction horizon that is larger than the controllability index can be sufficient for achieving a near-optimal performance, revealing a close relation between the prediction horizon and controllability. The obtained suboptimality performance bound is also applied to provide novel sample complexity and regret guarantees for nominal receding-horizon LQ controllers in a learning-based setting.
Related papers
- Accelerated zero-order SGD under high-order smoothness and overparameterized regime [79.85163929026146]
We present a novel gradient-free algorithm to solve convex optimization problems.
Such problems are encountered in medicine, physics, and machine learning.
We provide convergence guarantees for the proposed algorithm under both types of noise.
arXiv Detail & Related papers (2024-11-21T10:26:17Z) - Target Networks and Over-parameterization Stabilize Off-policy Bootstrapping with Function Approximation [47.08004958269403]
We prove a weaker convergence condition for bootstrapped value estimation in certain cases, even with off-policy data.
We extend our results to learning with truncated trajectories, showing that convergence is achievable for all tasks with minor modifications.
arXiv Detail & Related papers (2024-05-31T17:36:16Z) - Dimensionality Collapse: Optimal Measurement Selection for Low-Error
Infinite-Horizon Forecasting [3.5788754401889022]
We solve the problem of sequential linear measurement design as an infinite-horizon problem with the time-averaged trace of the Cram'er-Rao lower bound (CRLB) for forecasting as the cost.
By introducing theoretical results regarding measurements under additive noise from natural exponential families, we construct an equivalent problem from which a local dimensionality reduction can be derived.
This alternative formulation is based on the future collapse of dimensionality inherent in the limiting behavior of many differential equations and can be directly observed in the low-rank structure of the CRLB for forecasting.
arXiv Detail & Related papers (2023-03-27T17:25:04Z) - Calibrating Segmentation Networks with Margin-based Label Smoothing [19.669173092632]
We provide a unifying constrained-optimization perspective of current state-of-the-art calibration losses.
These losses could be viewed as approximations of a linear penalty imposing equality constraints on logit distances.
We propose a simple and flexible generalization based on inequality constraints, which imposes a controllable margin on logit distances.
arXiv Detail & Related papers (2022-09-09T20:21:03Z) - Error-based Knockoffs Inference for Controlled Feature Selection [49.99321384855201]
We propose an error-based knockoff inference method by integrating the knockoff features, the error-based feature importance statistics, and the stepdown procedure together.
The proposed inference procedure does not require specifying a regression model and can handle feature selection with theoretical guarantees.
arXiv Detail & Related papers (2022-03-09T01:55:59Z) - Improving Generalization via Uncertainty Driven Perturbations [107.45752065285821]
We consider uncertainty-driven perturbations of the training data points.
Unlike loss-driven perturbations, uncertainty-guided perturbations do not cross the decision boundary.
We show that UDP is guaranteed to achieve the robustness margin decision on linear models.
arXiv Detail & Related papers (2022-02-11T16:22:08Z) - The Devil is in the Margin: Margin-based Label Smoothing for Network
Calibration [21.63888208442176]
In spite of the dominant performances of deep neural networks, recent works have shown that they are poorly calibrated.
We provide a unifying constrained-optimization perspective of current state-of-the-art calibration losses.
We propose a simple and flexible generalization based on inequality constraints, which imposes a controllable margin on logit distances.
arXiv Detail & Related papers (2021-11-30T14:21:47Z) - Regret-optimal Estimation and Control [52.28457815067461]
We show that the regret-optimal estimator and regret-optimal controller can be derived in state-space form.
We propose regret-optimal analogs of Model-Predictive Control (MPC) and the Extended KalmanFilter (EKF) for systems with nonlinear dynamics.
arXiv Detail & Related papers (2021-06-22T23:14:21Z) - Reinforcement Learning of the Prediction Horizon in Model Predictive
Control [1.536989504296526]
We propose to learn the optimal prediction horizon as a function of the state using reinforcement learning (RL)
We show how the RL learning problem can be formulated and test our method on two control tasks, showing clear improvements over the fixed horizon MPC scheme.
arXiv Detail & Related papers (2021-02-22T15:52:32Z) - Regret-Optimal Filtering [57.51328978669528]
We consider the problem of filtering in linear state-space models through the lens of regret optimization.
We formulate a novel criterion for filter design based on the concept of regret between the estimation error energy of a clairvoyant estimator.
We show that the regret-optimal estimator can be easily implemented by solving three Riccati equations and a single Lyapunov equation.
arXiv Detail & Related papers (2021-01-25T19:06:52Z) - Adaptive Control and Regret Minimization in Linear Quadratic Gaussian
(LQG) Setting [91.43582419264763]
We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty.
LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model.
arXiv Detail & Related papers (2020-03-12T19:56:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.