Robust Learning-Based Control via Bootstrapped Multiplicative Noise
- URL: http://arxiv.org/abs/2002.10069v3
- Date: Wed, 11 Aug 2021 22:21:56 GMT
- Title: Robust Learning-Based Control via Bootstrapped Multiplicative Noise
- Authors: Benjamin Gravell and Tyler Summers
- Abstract summary: We propose a robust adaptive control algorithm that explicitly incorporates such non-asymptotic uncertainties into the control design.
A key advantage of the proposed approach is that the system identification and robust control design procedures both use uncertainty representations.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Despite decades of research and recent progress in adaptive control and
reinforcement learning, there remains a fundamental lack of understanding in
designing controllers that provide robustness to inherent non-asymptotic
uncertainties arising from models estimated with finite, noisy data. We propose
a robust adaptive control algorithm that explicitly incorporates such
non-asymptotic uncertainties into the control design. The algorithm has three
components: (1) a least-squares nominal model estimator; (2) a bootstrap
resampling method that quantifies non-asymptotic variance of the nominal model
estimate; and (3) a non-conventional robust control design method using an
optimal linear quadratic regulator (LQR) with multiplicative noise. A key
advantage of the proposed approach is that the system identification and robust
control design procedures both use stochastic uncertainty representations, so
that the actual inherent statistical estimation uncertainty directly aligns
with the uncertainty the robust controller is being designed against. We show
through numerical experiments that the proposed robust adaptive controller can
significantly outperform the certainty equivalent controller on both expected
regret and measures of regret risk.
Related papers
- Automatically Adaptive Conformal Risk Control [49.95190019041905]
We propose a methodology for achieving approximate conditional control of statistical risks by adapting to the difficulty of test samples.
Our framework goes beyond traditional conditional risk control based on user-provided conditioning events to the algorithmic, data-driven determination of appropriate function classes for conditioning.
arXiv Detail & Related papers (2024-06-25T08:29:32Z) - Adaptive Robust Model Predictive Control via Uncertainty Cancellation [25.736296938185074]
We propose a learning-based robust predictive control algorithm that compensates for significant uncertainty in the dynamics.
We optimize over a class of nonlinear feedback policies inspired by certainty equivalent "estimate-and-cancel" control laws.
arXiv Detail & Related papers (2022-12-02T18:54:23Z) - Probabilities Are Not Enough: Formal Controller Synthesis for Stochastic
Dynamical Models with Epistemic Uncertainty [68.00748155945047]
Capturing uncertainty in models of complex dynamical systems is crucial to designing safe controllers.
Several approaches use formal abstractions to synthesize policies that satisfy temporal specifications related to safety and reachability.
Our contribution is a novel abstraction-based controller method for continuous-state models with noise, uncertain parameters, and external disturbances.
arXiv Detail & Related papers (2022-10-12T07:57:03Z) - Robust Data-Driven Output Feedback Control via Bootstrapped
Multiplicative Noise [1.0312968200748118]
We propose a robust data-driven output feedback control algorithm that explicitly incorporates inherent finite-sample model estimate uncertainties into the control design.
A key advantage of the proposed approach is that the system identification and robust control design procedures both use uncertainty representations.
We show through numerical experiments that the proposed robust data-driven output feedback controller can significantly outperform a certainty equivalent controller.
arXiv Detail & Related papers (2022-05-10T18:47:14Z) - Regret-optimal Estimation and Control [52.28457815067461]
We show that the regret-optimal estimator and regret-optimal controller can be derived in state-space form.
We propose regret-optimal analogs of Model-Predictive Control (MPC) and the Extended KalmanFilter (EKF) for systems with nonlinear dynamics.
arXiv Detail & Related papers (2021-06-22T23:14:21Z) - Probabilistic robust linear quadratic regulators with Gaussian processes [73.0364959221845]
Probabilistic models such as Gaussian processes (GPs) are powerful tools to learn unknown dynamical systems from data for subsequent use in control design.
We present a novel controller synthesis for linearized GP dynamics that yields robust controllers with respect to a probabilistic stability margin.
arXiv Detail & Related papers (2021-05-17T08:36:18Z) - Adaptive Robust Model Predictive Control with Matched and Unmatched
Uncertainty [28.10549712956161]
We propose a learning-based robust predictive control algorithm that can handle large uncertainty in the dynamics for a class of discrete-time systems.
Motivated by an inability of existing learning-based predictive control algorithms to achieve safety guarantees in the presence of uncertainties of large magnitude, we achieve significant performance improvements.
arXiv Detail & Related papers (2021-04-16T17:47:02Z) - A Regret Minimization Approach to Iterative Learning Control [61.37088759497583]
We propose a new performance metric, planning regret, which replaces the standard uncertainty assumptions with worst case regret.
We provide theoretical and empirical evidence that the proposed algorithm outperforms existing methods on several benchmarks.
arXiv Detail & Related papers (2021-02-26T13:48:49Z) - Adaptive Control and Regret Minimization in Linear Quadratic Gaussian
(LQG) Setting [91.43582419264763]
We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty.
LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model.
arXiv Detail & Related papers (2020-03-12T19:56:38Z) - Optimistic robust linear quadratic dual control [4.94950858749529]
We present a dual control strategy that attempts to combine the performance of certainty equivalence, with the practical utility of robustness.
The formulation preserves structure in the representation of parametric uncertainty, which allows the controller to target reduction of uncertainty in the parameters that matter most' for the control task.
arXiv Detail & Related papers (2019-12-31T02:02:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.