Adaptive Model Predictive Control by Learning Classifiers
- URL: http://arxiv.org/abs/2203.06783v1
- Date: Sun, 13 Mar 2022 23:22:12 GMT
- Title: Adaptive Model Predictive Control by Learning Classifiers
- Authors: Rel Guzman, Rafael Oliveira, Fabio Ramos
- Abstract summary: We propose an adaptive MPC variant that automatically estimates control and model parameters.
We leverage recent results showing that BO can be formulated as a density ratio estimation.
This is then integrated into a model predictive path integral control framework yielding robust controllers for a variety of challenging robotics tasks.
- Score: 26.052368583196426
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Stochastic model predictive control has been a successful and robust control
framework for many robotics tasks where the system dynamics model is slightly
inaccurate or in the presence of environment disturbances. Despite the
successes, it is still unclear how to best adjust control parameters to the
current task in the presence of model parameter uncertainty and heteroscedastic
noise. In this paper, we propose an adaptive MPC variant that automatically
estimates control and model parameters by leveraging ideas from Bayesian
optimization (BO) and the classical expected improvement acquisition function.
We leverage recent results showing that BO can be formulated as a density ratio
estimation which can be efficiently approximated by simply learning a
classifier. This is then integrated into a model predictive path integral
control framework yielding robust controllers for a variety of challenging
robotics tasks. We demonstrate the approach on classical control problems under
model uncertainty and robotics manipulation tasks.
Related papers
- Tuning Legged Locomotion Controllers via Safe Bayesian Optimization [47.87675010450171]
This paper presents a data-driven strategy to streamline the deployment of model-based controllers in legged robotic hardware platforms.
We leverage a model-free safe learning algorithm to automate the tuning of control gains, addressing the mismatch between the simplified model used in the control formulation and the real system.
arXiv Detail & Related papers (2023-06-12T13:10:14Z) - Predictable MDP Abstraction for Unsupervised Model-Based RL [93.91375268580806]
We propose predictable MDP abstraction (PMA)
Instead of training a predictive model on the original MDP, we train a model on a transformed MDP with a learned action space.
We theoretically analyze PMA and empirically demonstrate that PMA leads to significant improvements over prior unsupervised model-based RL approaches.
arXiv Detail & Related papers (2023-02-08T07:37:51Z) - Active Learning of Discrete-Time Dynamics for Uncertainty-Aware Model Predictive Control [46.81433026280051]
We present a self-supervised learning approach that actively models the dynamics of nonlinear robotic systems.
Our approach showcases high resilience and generalization capabilities by consistently adapting to unseen flight conditions.
arXiv Detail & Related papers (2022-10-23T00:45:05Z) - Bayesian Optimisation for Robust Model Predictive Control under Model
Parameter Uncertainty [26.052368583196426]
We propose an adaptive optimisation approach for tuning model predictive control (MPC) hyper- parameters.
We develop a Bayesian optimisation (BO) algorithm with a heteroscedastic noise model to deal with varying noise.
Experimental results demonstrate that our approach leads to higher cumulative rewards and more stable controllers.
arXiv Detail & Related papers (2022-03-01T15:33:21Z) - Sample-Efficient Reinforcement Learning via Conservative Model-Based
Actor-Critic [67.00475077281212]
Model-based reinforcement learning algorithms are more sample efficient than their model-free counterparts.
We propose a novel approach that achieves high sample efficiency without the strong reliance on accurate learned models.
We show that CMBAC significantly outperforms state-of-the-art approaches in terms of sample efficiency on several challenging tasks.
arXiv Detail & Related papers (2021-12-16T15:33:11Z) - Evaluating model-based planning and planner amortization for continuous
control [79.49319308600228]
We take a hybrid approach, combining model predictive control (MPC) with a learned model and model-free policy learning.
We find that well-tuned model-free agents are strong baselines even for high DoF control problems.
We show that it is possible to distil a model-based planner into a policy that amortizes the planning without any loss of performance.
arXiv Detail & Related papers (2021-10-07T12:00:40Z) - Safe and Efficient Model-free Adaptive Control via Bayesian Optimization [39.962395119933596]
We propose a purely data-driven, model-free approach for adaptive control.
tuning low-level controllers based solely on system data raises concerns on the underlying algorithm safety and computational performance.
We numerically demonstrate for several types of disturbances that our approach is sample efficient, outperforms constrained Bayesian optimization in terms of safety, and achieves the performance optima computed by grid evaluation.
arXiv Detail & Related papers (2021-01-19T19:15:00Z) - Heteroscedastic Bayesian Optimisation for Stochastic Model Predictive
Control [23.180330602334223]
Model predictive control (MPC) has been successful in applications involving the control of complex physical systems.
We investigate fine-tuning MPC methods in the context of MPC, which presents extra challenges due to the randomness of the controller's actions.
arXiv Detail & Related papers (2020-10-01T05:31:41Z) - Control as Hybrid Inference [62.997667081978825]
We present an implementation of CHI which naturally mediates the balance between iterative and amortised inference.
We verify the scalability of our algorithm on a continuous control benchmark, demonstrating that it outperforms strong model-free and model-based baselines.
arXiv Detail & Related papers (2020-07-11T19:44:09Z) - Adaptive Control and Regret Minimization in Linear Quadratic Gaussian
(LQG) Setting [91.43582419264763]
We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty.
LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model.
arXiv Detail & Related papers (2020-03-12T19:56:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.