Regret-optimal Estimation and Control
- URL: http://arxiv.org/abs/2106.12097v1
- Date: Tue, 22 Jun 2021 23:14:21 GMT
- Title: Regret-optimal Estimation and Control
- Authors: Gautam Goel, Babak Hassibi
- Abstract summary: We show that the regret-optimal estimator and regret-optimal controller can be derived in state-space form.
We propose regret-optimal analogs of Model-Predictive Control (MPC) and the Extended KalmanFilter (EKF) for systems with nonlinear dynamics.
- Score: 52.28457815067461
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We consider estimation and control in linear time-varying dynamical systems
from the perspective of regret minimization. Unlike most prior work in this
area, we focus on the problem of designing causal estimators and controllers
which compete against a clairvoyant noncausal policy, instead of the best
policy selected in hindsight from some fixed parametric class. We show that the
regret-optimal estimator and regret-optimal controller can be derived in
state-space form using operator-theoretic techniques from robust control and
present tight,data-dependent bounds on the regret incurred by our algorithms in
terms of the energy of the disturbances. Our results can be viewed as extending
traditional robust estimation and control, which focuses on minimizing
worst-case cost, to minimizing worst-case regret. We propose regret-optimal
analogs of Model-Predictive Control (MPC) and the Extended KalmanFilter (EKF)
for systems with nonlinear dynamics and present numerical experiments which
show that our regret-optimal algorithms can significantly outperform standard
approaches to estimation and control.
Related papers
- Growing Q-Networks: Solving Continuous Control Tasks with Adaptive Control Resolution [51.83951489847344]
In robotics applications, smooth control signals are commonly preferred to reduce system wear and energy efficiency.
In this work, we aim to bridge this performance gap by growing discrete action spaces from coarse to fine control resolution.
Our work indicates that an adaptive control resolution in combination with value decomposition yields simple critic-only algorithms that yield surprisingly strong performance on continuous control tasks.
arXiv Detail & Related papers (2024-04-05T17:58:37Z) - Sub-linear Regret in Adaptive Model Predictive Control [56.705978425244496]
We present STT-MPC (Self-Tuning Tube-based Model Predictive Control), an online oracle that combines the certainty-equivalence principle and polytopic tubes.
We analyze the regret of the algorithm, when compared to an algorithm initially aware of the system dynamics.
arXiv Detail & Related papers (2023-10-07T15:07:10Z) - Follow the Clairvoyant: an Imitation Learning Approach to Optimal
Control [4.978565634673048]
We consider control of dynamical systems through the lens of competitive analysis.
Motivated by the observation that the optimal cost only provides coarse information about the ideal closed-loop behavior, we propose directly minimizing the tracking error.
arXiv Detail & Related papers (2022-11-14T14:15:12Z) - Steady-State Error Compensation in Reference Tracking and Disturbance
Rejection Problems for Reinforcement Learning-Based Control [0.9023847175654602]
Reinforcement learning (RL) is a promising, upcoming topic in automatic control applications.
Initiative action state augmentation (IASA) for actor-critic-based RL controllers is introduced.
This augmentation does not require any expert knowledge, leaving the approach model free.
arXiv Detail & Related papers (2022-01-31T16:29:19Z) - Regret Analysis of Learning-Based MPC with Partially-Unknown Cost
Function [5.601217969637838]
exploration/exploitation trade-off is an inherent challenge in data-driven and adaptive control.
We propose the use of a finitehorizon oracle controller with perfect knowledge of all system parameters as a reference for optimal control actions.
We develop learning-based policies that we prove achieve low regret with respect to this oracle finite-horizon controller.
arXiv Detail & Related papers (2021-08-04T22:43:51Z) - Non-stationary Online Learning with Memory and Non-stochastic Control [71.14503310914799]
We study the problem of Online Convex Optimization (OCO) with memory, which allows loss functions to depend on past decisions.
In this paper, we introduce dynamic policy regret as the performance measure to design algorithms robust to non-stationary environments.
We propose a novel algorithm for OCO with memory that provably enjoys an optimal dynamic policy regret in terms of time horizon, non-stationarity measure, and memory length.
arXiv Detail & Related papers (2021-02-07T09:45:15Z) - Gaussian Process-based Min-norm Stabilizing Controller for
Control-Affine Systems with Uncertain Input Effects and Dynamics [90.81186513537777]
We propose a novel compound kernel that captures the control-affine nature of the problem.
We show that this resulting optimization problem is convex, and we call it Gaussian Process-based Control Lyapunov Function Second-Order Cone Program (GP-CLF-SOCP)
arXiv Detail & Related papers (2020-11-14T01:27:32Z) - Regret-optimal control in dynamic environments [39.76359052907755]
We focus on the problem of designing an online controller which minimizes regret against the best dynamic sequence of control actions selected in hindsight.
We derive the state-space structure of the regret-optimal controller via a novel reduction to $H_infty$ control.
We present numerical experiments which show that our regret-optimal controller interpolates between the performance of the $H_infty$-optimal controllers across and adversarial environments.
arXiv Detail & Related papers (2020-10-20T17:32:17Z) - Adaptive Control and Regret Minimization in Linear Quadratic Gaussian
(LQG) Setting [91.43582419264763]
We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty.
LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model.
arXiv Detail & Related papers (2020-03-12T19:56:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.