Optimization of the Model Predictive Control Meta-Parameters Through Reinforcement Learning

• URL: http://arxiv.org/abs/2111.04146v1
• Date: Sun, 7 Nov 2021 18:33:22 GMT
• Title: Optimization of the Model Predictive Control Meta-Parameters Through Reinforcement Learning
• Authors: Eivind B{\o}hn, Sebastien Gros, Signe Moe, and Tor Arne Johansen
• Abstract summary: We propose a novel framework in which any parameter of the control algorithm can be jointly tuned using reinforcement learning(RL) We demonstrate our framework on the inverted pendulum control task, reducing the total time of the control system by 36% while also improving the control performance by 18.4% over the best-performing MPC baseline.
• Score: 1.4069478981641936
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Model predictive control (MPC) is increasingly being considered for control of fast systems and embedded applications. However, the MPC has some significant challenges for such systems. Its high computational complexity results in high power consumption from the control algorithm, which could account for a significant share of the energy resources in battery-powered embedded systems. The MPC parameters must be tuned, which is largely a trial-and-error process that affects the control performance, the robustness and the computational complexity of the controller to a high degree. In this paper, we propose a novel framework in which any parameter of the control algorithm can be jointly tuned using reinforcement learning(RL), with the goal of simultaneously optimizing the control performance and the power usage of the control algorithm. We propose the novel idea of optimizing the meta-parameters of MPCwith RL, i.e. parameters affecting the structure of the MPCproblem as opposed to the solution to a given problem. Our control algorithm is based on an event-triggered MPC where we learn when the MPC should be re-computed, and a dual mode MPC and linear state feedback control law applied in between MPC computations. We formulate a novel mixture-distribution policy and show that with joint optimization we achieve improvements that do not present themselves when optimizing the same parameters in isolation. We demonstrate our framework on the inverted pendulum control task, reducing the total computation time of the control system by 36% while also improving the control performance by 18.4% over the best-performing MPC baseline.

Related papers

• Comparison of Model Predictive Control and Proximal Policy Optimization for a 1-DOF Helicopter System [0.7499722271664147]
  This study conducts a comparative analysis of Model Predictive Control (MPC) and Proximal Policy Optimization (PPO), a Deep Reinforcement Learning (DRL) algorithm, applied to a Quanser Aero 2 system. PPO excels in rise-time and adaptability, making it a promising approach for applications requiring rapid response and adaptability.
  arXiv  Detail & Related papers  (2024-08-28T08:35:34Z)
• Parameter-Adaptive Approximate MPC: Tuning Neural-Network Controllers without Retraining [50.00291020618743]
  This work introduces a novel, parameter-adaptive AMPC architecture capable of online tuning without recomputing large datasets and retraining. We showcase the effectiveness of parameter-adaptive AMPC by controlling the swing-ups of two different real cartpole systems with a severely resource-constrained microcontroller (MCU) Taken together, these contributions represent a marked step toward the practical application of AMPC in real-world systems.
  arXiv  Detail & Related papers  (2024-04-08T20:02:19Z)
• MPC-Inspired Reinforcement Learning for Verifiable Model-Free Control [5.9867297878688195]
  We introduce a new class of parameterized controllers, drawing inspiration from Model Predictive Control (MPC) The controller resembles a Quadratic Programming (QP) solver of a linear MPC problem, with the parameters of the controller being trained via Deep Reinforcement Learning (DRL) The proposed controller is significantly more computationally efficient compared to MPC and requires fewer parameters to learn than controllers.
  arXiv  Detail & Related papers  (2023-12-08T19:33:22Z)
• An Automatic Tuning MPC with Application to Ecological Cruise Control [0.0]
  We show an approach for online automatic tuning of an MPC controller with an example application to an ecological cruise control system. We solve the global fuel consumption minimization problem offline using dynamic programming and find the corresponding MPC cost function. A neural network fitted to these offline results is used to generate the desired MPC cost function weight during online operation.
  arXiv  Detail & Related papers  (2023-09-17T19:49:47Z)
• Collaborative Intelligent Reflecting Surface Networks with Multi-Agent Reinforcement Learning [63.83425382922157]
  Intelligent reflecting surface (IRS) is envisioned to be widely applied in future wireless networks. In this paper, we investigate a multi-user communication system assisted by cooperative IRS devices with the capability of energy harvesting.
  arXiv  Detail & Related papers  (2022-03-26T20:37:14Z)
• Policy Search for Model Predictive Control with Application to Agile Drone Flight [56.24908013905407]
  We propose a policy-search-for-model-predictive-control framework for MPC. Specifically, we formulate the MPC as a parameterized controller, where the hard-to-optimize decision variables are represented as high-level policies. Experiments show that our controller achieves robust and real-time control performance in both simulation and the real world.
  arXiv  Detail & Related papers  (2021-12-07T17:39:24Z)
• Covert Model Poisoning Against Federated Learning: Algorithm Design and Optimization [76.51980153902774]
  Federated learning (FL) is vulnerable to external attacks on FL models during parameters transmissions. In this paper, we propose effective MP algorithms to combat state-of-the-art defensive aggregation mechanisms. Our experimental results demonstrate that the proposed CMP algorithms are effective and substantially outperform existing attack mechanisms.
  arXiv  Detail & Related papers  (2021-01-28T03:28:18Z)
• Optimization of the Model Predictive Control Update Interval Using Reinforcement Learning [0.7952582509792969]
  In control applications there is often a compromise that needs to be made with regards to the complexity and performance of the controller. We propose a controller architecture in which the computational cost is explicitly optimized along with the control objective.
  arXiv  Detail & Related papers  (2020-11-26T16:01:52Z)
• Heteroscedastic Bayesian Optimisation for Stochastic Model Predictive Control [23.180330602334223]
  Model predictive control (MPC) has been successful in applications involving the control of complex physical systems. We investigate fine-tuning MPC methods in the context of MPC, which presents extra challenges due to the randomness of the controller's actions.
  arXiv  Detail & Related papers  (2020-10-01T05:31:41Z)
• Learning High-Level Policies for Model Predictive Control [54.00297896763184]
  Model Predictive Control (MPC) provides robust solutions to robot control tasks. We propose a self-supervised learning algorithm for learning a neural network high-level policy. We show that our approach can handle situations that are difficult for standard MPC.
  arXiv  Detail & Related papers  (2020-07-20T17:12:34Z)
• Information Theoretic Model Predictive Q-Learning [64.74041985237105]
  We present a novel theoretical connection between information theoretic MPC and entropy regularized RL. We develop a Q-learning algorithm that can leverage biased models.
  arXiv  Detail & Related papers  (2019-12-31T00:29:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.