Adaptive Model Predictive Control by Learning Classifiers
- URL: http://arxiv.org/abs/2203.06783v1
- Date: Sun, 13 Mar 2022 23:22:12 GMT
- Title: Adaptive Model Predictive Control by Learning Classifiers
- Authors: Rel Guzman, Rafael Oliveira, Fabio Ramos
- Abstract summary: We propose an adaptive MPC variant that automatically estimates control and model parameters.
We leverage recent results showing that BO can be formulated as a density ratio estimation.
This is then integrated into a model predictive path integral control framework yielding robust controllers for a variety of challenging robotics tasks.
- Score: 26.052368583196426
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Stochastic model predictive control has been a successful and robust control
framework for many robotics tasks where the system dynamics model is slightly
inaccurate or in the presence of environment disturbances. Despite the
successes, it is still unclear how to best adjust control parameters to the
current task in the presence of model parameter uncertainty and heteroscedastic
noise. In this paper, we propose an adaptive MPC variant that automatically
estimates control and model parameters by leveraging ideas from Bayesian
optimization (BO) and the classical expected improvement acquisition function.
We leverage recent results showing that BO can be formulated as a density ratio
estimation which can be efficiently approximated by simply learning a
classifier. This is then integrated into a model predictive path integral
control framework yielding robust controllers for a variety of challenging
robotics tasks. We demonstrate the approach on classical control problems under
model uncertainty and robotics manipulation tasks.
Related papers
- Actively Learning Reinforcement Learning: A Stochastic Optimal Control
Approach [3.7728340443952577]
We propose a framework towards achieving two intertwined objectives: (i) equipping reinforcement learning with active exploration and deliberate information gathering, such that it regulates state and uncertainties resulting from modeling mismatches and noisy sensory; and (ii) overcoming the huge computational cost of optimal control.
We approach both objectives by using reinforcement learning to attain the optimal control law.
arXiv Detail & Related papers (2023-09-18T18:05:35Z) - Tuning Legged Locomotion Controllers via Safe Bayesian Optimization [47.87675010450171]
This paper presents a data-driven strategy to streamline the deployment of model-based controllers in legged robotic hardware platforms.
We leverage a model-free safe learning algorithm to automate the tuning of control gains, addressing the mismatch between the simplified model used in the control formulation and the real system.
arXiv Detail & Related papers (2023-06-12T13:10:14Z) - Active Learning of Discrete-Time Dynamics for Uncertainty-Aware Model
Predictive Control [49.60520501097199]
We present a self-supervised learning approach that actively models the dynamics of nonlinear robotic systems.
Our approach showcases high resilience and generalization capabilities by consistently adapting to unseen flight conditions.
arXiv Detail & Related papers (2022-10-23T00:45:05Z) - Bayesian Optimisation for Robust Model Predictive Control under Model
Parameter Uncertainty [26.052368583196426]
We propose an adaptive optimisation approach for tuning model predictive control (MPC) hyper- parameters.
We develop a Bayesian optimisation (BO) algorithm with a heteroscedastic noise model to deal with varying noise.
Experimental results demonstrate that our approach leads to higher cumulative rewards and more stable controllers.
arXiv Detail & Related papers (2022-03-01T15:33:21Z) - Sample-Efficient Reinforcement Learning via Conservative Model-Based
Actor-Critic [67.00475077281212]
Model-based reinforcement learning algorithms are more sample efficient than their model-free counterparts.
We propose a novel approach that achieves high sample efficiency without the strong reliance on accurate learned models.
We show that CMBAC significantly outperforms state-of-the-art approaches in terms of sample efficiency on several challenging tasks.
arXiv Detail & Related papers (2021-12-16T15:33:11Z) - Evaluating model-based planning and planner amortization for continuous
control [79.49319308600228]
We take a hybrid approach, combining model predictive control (MPC) with a learned model and model-free policy learning.
We find that well-tuned model-free agents are strong baselines even for high DoF control problems.
We show that it is possible to distil a model-based planner into a policy that amortizes the planning without any loss of performance.
arXiv Detail & Related papers (2021-10-07T12:00:40Z) - Heteroscedastic Bayesian Optimisation for Stochastic Model Predictive
Control [23.180330602334223]
Model predictive control (MPC) has been successful in applications involving the control of complex physical systems.
We investigate fine-tuning MPC methods in the context of MPC, which presents extra challenges due to the randomness of the controller's actions.
arXiv Detail & Related papers (2020-10-01T05:31:41Z) - Control as Hybrid Inference [62.997667081978825]
We present an implementation of CHI which naturally mediates the balance between iterative and amortised inference.
We verify the scalability of our algorithm on a continuous control benchmark, demonstrating that it outperforms strong model-free and model-based baselines.
arXiv Detail & Related papers (2020-07-11T19:44:09Z) - Adaptive Control and Regret Minimization in Linear Quadratic Gaussian
(LQG) Setting [91.43582419264763]
We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty.
LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model.
arXiv Detail & Related papers (2020-03-12T19:56:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.