Optimal Competitive-Ratio Control
- URL: http://arxiv.org/abs/2206.01782v1
- Date: Fri, 3 Jun 2022 19:01:07 GMT
- Title: Optimal Competitive-Ratio Control
- Authors: Oron Sabag, Sahin Lale, Babak Hassibi
- Abstract summary: We show that the optimal competitive ratio formula can be computed as the maximal eigenvalue of a simple matrix.
We conduct an extensive numerical study to verify this analytical solution, and demonstrate that the optimal competitive-ratio controller outperforms other controllers on several large scale practical systems.
- Score: 40.89951305613357
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Inspired by competitive policy designs approaches in online learning, new
control paradigms such as competitive-ratio and regret-optimal control have
been recently proposed as alternatives to the classical $\mathcal{H}_2$ and
$\mathcal{H}_\infty$ approaches. These competitive metrics compare the control
cost of the designed controller against the cost of a clairvoyant controller,
which has access to past, present, and future disturbances in terms of ratio
and difference, respectively. While prior work provided the optimal solution
for the regret-optimal control problem, in competitive-ratio control, the
solution is only provided for the sub-optimal problem. In this work, we derive
the optimal solution to the competitive-ratio control problem. We show that the
optimal competitive ratio formula can be computed as the maximal eigenvalue of
a simple matrix, and provide a state-space controller that achieves the optimal
competitive ratio. We conduct an extensive numerical study to verify this
analytical solution, and demonstrate that the optimal competitive-ratio
controller outperforms other controllers on several large scale practical
systems. The key techniques that underpin our explicit solution is a reduction
of the control problem to a Nehari problem, along with a novel factorization of
the clairvoyant controller's cost. We reveal an interesting relation between
the explicit solutions that now exist for both competitive control paradigms by
formulating a regret-optimal control framework with weight functions that can
also be utilized for practical purposes.
Related papers
- Growing Q-Networks: Solving Continuous Control Tasks with Adaptive Control Resolution [51.83951489847344]
In robotics applications, smooth control signals are commonly preferred to reduce system wear and energy efficiency.
In this work, we aim to bridge this performance gap by growing discrete action spaces from coarse to fine control resolution.
Our work indicates that an adaptive control resolution in combination with value decomposition yields simple critic-only algorithms that yield surprisingly strong performance on continuous control tasks.
arXiv Detail & Related papers (2024-04-05T17:58:37Z) - Reinforcement Learning-based Receding Horizon Control using Adaptive Control Barrier Functions for Safety-Critical Systems [14.166970599802324]
Optimal control methods provide solutions to safety-critical problems but easily become intractable.
We propose a Reinforcement Learning-based Receding Horizon Control approach leveraging Model Predictive Control.
We validate our method by applying it to the challenging automated merging control problem for Connected and Automated Vehicles.
arXiv Detail & Related papers (2024-03-26T02:49:08Z) - Follow the Clairvoyant: an Imitation Learning Approach to Optimal
Control [4.978565634673048]
We consider control of dynamical systems through the lens of competitive analysis.
Motivated by the observation that the optimal cost only provides coarse information about the ideal closed-loop behavior, we propose directly minimizing the tracking error.
arXiv Detail & Related papers (2022-11-14T14:15:12Z) - Competitive Control [52.28457815067461]
We focus on designing an online controller which competes against a clairvoyant offline optimal controller.
A natural performance metric in this setting is competitive ratio, which is the ratio between the cost incurred by the online controller and the cost incurred by the offline optimal controller.
arXiv Detail & Related papers (2021-07-28T22:26:27Z) - Regret-optimal Estimation and Control [52.28457815067461]
We show that the regret-optimal estimator and regret-optimal controller can be derived in state-space form.
We propose regret-optimal analogs of Model-Predictive Control (MPC) and the Extended KalmanFilter (EKF) for systems with nonlinear dynamics.
arXiv Detail & Related papers (2021-06-22T23:14:21Z) - Stable Online Control of Linear Time-Varying Systems [49.41696101740271]
COCO-LQ is an efficient online control algorithm that guarantees input-to-state stability for a large class of LTV systems.
We empirically demonstrate the performance of COCO-LQ in both synthetic experiments and a power system frequency control example.
arXiv Detail & Related papers (2021-04-29T06:18:49Z) - Non-stationary Online Learning with Memory and Non-stochastic Control [71.14503310914799]
We study the problem of Online Convex Optimization (OCO) with memory, which allows loss functions to depend on past decisions.
In this paper, we introduce dynamic policy regret as the performance measure to design algorithms robust to non-stationary environments.
We propose a novel algorithm for OCO with memory that provably enjoys an optimal dynamic policy regret in terms of time horizon, non-stationarity measure, and memory length.
arXiv Detail & Related papers (2021-02-07T09:45:15Z) - Regret-optimal measurement-feedback control [39.76359052907755]
We consider measurement-feedback control in linear dynamical systems from the perspective of regret.
We show that in the measurement-feedback setting, unlike in the full information setting, there is no single offline controller which outperforms every other offline controller on every disturbance.
We show that the corresponding regret-optimal online controller can be found via a novel reduction to the classical Nehari problem and present a tight data-dependent bound on its regret.
arXiv Detail & Related papers (2020-11-24T01:36:48Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.