Efficient Online Linear Control with Stochastic Convex Costs and Unknown Dynamics

• URL: http://arxiv.org/abs/2203.01170v1
• Date: Wed, 2 Mar 2022 15:19:20 GMT
• Title: Efficient Online Linear Control with Stochastic Convex Costs and Unknown Dynamics
• Authors: Asaf Cassel (1), Alon Cohen (2 and 3), Tomer Koren (1 and 3) ((1) School of Computer Science, Tel Aviv University, (2) School of Electrical Engineering, Tel Aviv University, (3) Google Research, Tel Aviv)
• Abstract summary: We present a computationally efficient algorithm that attains an optimal $sqrtT$ regret-rate against the best stabilizing linear controller. In contrast to previous work, our algorithm is based on the Optimism in the Face of Uncertainty paradigm.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We consider the problem of controlling an unknown linear dynamical system under a stochastic convex cost and full feedback of both the state and cost function. We present a computationally efficient algorithm that attains an optimal $\sqrt{T}$ regret-rate against the best stabilizing linear controller. In contrast to previous work, our algorithm is based on the Optimism in the Face of Uncertainty paradigm. This results in a substantially improved computational complexity and a simpler analysis.

Related papers

• First-Order Dynamic Optimization for Streaming Convex Costs [0.0]
  We develop an approach to track the optimal solution with a bounded error. Our algorithm is executed only by using the first-order derivatives of the cost function.
  arXiv  Detail & Related papers  (2023-10-11T22:41:00Z)
• Sub-linear Regret in Adaptive Model Predictive Control [56.705978425244496]
  We present STT-MPC (Self-Tuning Tube-based Model Predictive Control), an online oracle that combines the certainty-equivalence principle and polytopic tubes. We analyze the regret of the algorithm, when compared to an algorithm initially aware of the system dynamics.
  arXiv  Detail & Related papers  (2023-10-07T15:07:10Z)
• On Convex Data-Driven Inverse Optimal Control for Nonlinear, Non-stationary and Stochastic Systems [0.7240153598817866]
  This paper is concerned with a finite-horizon inverse control problem, which has the goal of reconstructing, from observations, the possibly non-nonstationary cost driving the actions of an agent. We present a result enabling cost optimization by solving an problem that is convex non-stationary agent cost. All experiments confirm the effectiveness of our approach.
  arXiv  Detail & Related papers  (2023-06-24T10:25:53Z)
• Exploring the Algorithm-Dependent Generalization of AUPRC Optimization with List Stability [107.65337427333064]
  optimization of the Area Under the Precision-Recall Curve (AUPRC) is a crucial problem for machine learning. In this work, we present the first trial in the single-dependent generalization of AUPRC optimization. Experiments on three image retrieval datasets on speak to the effectiveness and soundness of our framework.
  arXiv  Detail & Related papers  (2022-09-27T09:06:37Z)
• On the Optimization Landscape of Dynamic Output Feedback: A Case Study for Linear Quadratic Regulator [12.255864026960403]
  We show how the dLQR cost varies with the coordinate transformation of the dynamic controller and then derive the optimal transformation for a given observable stabilizing controller. These results shed light on designing efficient algorithms for general decision-making problems with partially observed information.
  arXiv  Detail & Related papers  (2022-09-12T06:43:35Z)
• Rate-Optimal Online Convex Optimization in Adaptive Linear Control [0.0]
  We consider the problem of controlling an unknown convex linear system under adversarially changing costs. We present the first computationally-gret that attains an optimal linear hindsight function.
  arXiv  Detail & Related papers  (2022-06-03T07:32:11Z)
• Robust Online Control with Model Misspecification [96.23493624553998]
  We study online control of an unknown nonlinear dynamical system with model misspecification. Our study focuses on robustness, which measures how much deviation from the assumed linear approximation can be tolerated.
  arXiv  Detail & Related papers  (2021-07-16T07:04:35Z)
• Efficient Optimistic Exploration in Linear-Quadratic Regulators via Lagrangian Relaxation [107.06364966905821]
  We study the exploration-exploitation dilemma in the linear quadratic regulator (LQR) setting. Inspired by the extended value iteration algorithm used in optimistic algorithms for finite MDPs, we propose to relax the optimistic optimization of ofulq. We show that an $epsilon$-optimistic controller can be computed efficiently by solving at most $Obig(log (1/epsilon)big)$ Riccati equations.
  arXiv  Detail & Related papers  (2020-07-13T16:30:47Z)
• Adaptive Control and Regret Minimization in Linear Quadratic Gaussian (LQG) Setting [91.43582419264763]
  We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty. LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model.
  arXiv  Detail & Related papers  (2020-03-12T19:56:38Z)
• Logarithmic Regret for Adversarial Online Control [56.12283443161479]
  We give the first algorithm with logarithmic regret for arbitrary adversarial disturbance sequences. Our algorithm and analysis use a characterization for the offline control law to reduce the online control problem to (delayed) online learning.
  arXiv  Detail & Related papers  (2020-02-29T06:29:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.