Lifelong Control of Off-grid Microgrid with Model Based Reinforcement Learning

• URL: http://arxiv.org/abs/2005.08006v1
• Date: Sat, 16 May 2020 14:45:55 GMT
• Title: Lifelong Control of Off-grid Microgrid with Model Based Reinforcement Learning
• Authors: Simone Totaro, Ioannis Boukas, Anders Jonsson and Bertrand Corn\'elusse
• Abstract summary: We present an open-source reinforcement framework for the modeling of an off-grid microgrid for rural electrification. We categorize the set of changes that can occur in progressive and abrupt changes. We propose a novel model based reinforcement learning algorithm that is able to address both types of changes.
• Score: 33.55238830808043
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The lifelong control problem of an off-grid microgrid is composed of two tasks, namely estimation of the condition of the microgrid devices and operational planning accounting for the uncertainties by forecasting the future consumption and the renewable production. The main challenge for the effective control arises from the various changes that take place over time. In this paper, we present an open-source reinforcement framework for the modeling of an off-grid microgrid for rural electrification. The lifelong control problem of an isolated microgrid is formulated as a Markov Decision Process (MDP). We categorize the set of changes that can occur in progressive and abrupt changes. We propose a novel model based reinforcement learning algorithm that is able to address both types of changes. In particular the proposed algorithm demonstrates generalisation properties, transfer capabilities and better robustness in case of fast-changing system dynamics. The proposed algorithm is compared against a rule-based policy and a model predictive controller with look-ahead. The results show that the trained agent is able to outperform both benchmarks in the lifelong setting where the system dynamics are changing over time.

Related papers

• Integrating Reinforcement Learning and Model Predictive Control with Applications to Microgrids [14.389086937116582]
  This work proposes an approach that integrates reinforcement learning and model predictive control (MPC) to solve optimal control problems in mixed-logical dynamical systems. The proposed method significantly reduces the online computation time of the MPC approach and that it generates policies with small optimality gaps and high feasibility rates.
  arXiv  Detail & Related papers  (2024-09-17T15:17:16Z)
• A Policy Iteration Approach for Flock Motion Control [5.419608513284392]
  The overall control process guides the agents while monitoring the flock-cohesiveness and localization. An online model-free policy iteration mechanism is developed here to guide a flock of agents to follow an independent command generator. The simulation results of the policy iteration mechanism revealed fast learning and convergence behaviors with less computational effort.
  arXiv  Detail & Related papers  (2023-03-17T15:04:57Z)
• Incorporating Recurrent Reinforcement Learning into Model Predictive Control for Adaptive Control in Autonomous Driving [11.67417895998434]
  Model Predictive Control (MPC) is attracting tremendous attention in the autonomous driving task as a powerful control technique. In this paper, we reformulate the problem as a Partially Observed Markov Decision Process (POMDP) We then learn a recurrent policy continually adapting the parameters of the dynamics model via Recurrent Reinforcement Learning (RRL) for optimal and adaptive control.
  arXiv  Detail & Related papers  (2023-01-30T22:11:07Z)
• On Controller Tuning with Time-Varying Bayesian Optimization [74.57758188038375]
  We will use time-varying optimization (TVBO) to tune controllers online in changing environments using appropriate prior knowledge on the control objective and its changes. We propose a novel TVBO strategy using Uncertainty-Injection (UI), which incorporates the assumption of incremental and lasting changes. Our model outperforms the state-of-the-art method in TVBO, exhibiting reduced regret and fewer unstable parameter configurations.
  arXiv  Detail & Related papers  (2022-07-22T14:54:13Z)
• Adaptive Model Predictive Control by Learning Classifiers [26.052368583196426]
  We propose an adaptive MPC variant that automatically estimates control and model parameters. We leverage recent results showing that BO can be formulated as a density ratio estimation. This is then integrated into a model predictive path integral control framework yielding robust controllers for a variety of challenging robotics tasks.
  arXiv  Detail & Related papers  (2022-03-13T23:22:12Z)
• Machine Learning based Optimal Feedback Control for Microgrid Stabilization [6.035279357076201]
  An energy storage based feedback controller can compensate undesired dynamics of a microgrid to improve its stability. This paper proposes a machine learning-based optimal feedback control scheme. A case study is carried out for a microgrid model based on a modified Kundur two-area system to test the real-time performance of the proposed control scheme.
  arXiv  Detail & Related papers  (2022-03-09T15:44:56Z)
• Evaluating model-based planning and planner amortization for continuous control [79.49319308600228]
  We take a hybrid approach, combining model predictive control (MPC) with a learned model and model-free policy learning. We find that well-tuned model-free agents are strong baselines even for high DoF control problems. We show that it is possible to distil a model-based planner into a policy that amortizes the planning without any loss of performance.
  arXiv  Detail & Related papers  (2021-10-07T12:00:40Z)
• Anticipating the Long-Term Effect of Online Learning in Control [75.6527644813815]
  AntLer is a design algorithm for learning-based control laws that anticipates learning. We show that AntLer approximates an optimal solution arbitrarily accurately with probability one.
  arXiv  Detail & Related papers  (2020-07-24T07:00:14Z)
• Logarithmic Regret Bound in Partially Observable Linear Dynamical Systems [91.43582419264763]
  We study the problem of system identification and adaptive control in partially observable linear dynamical systems. We present the first model estimation method with finite-time guarantees in both open and closed-loop system identification. We show that AdaptOn is the first algorithm that achieves $textpolylogleft(Tright)$ regret in adaptive control of unknown partially observable linear dynamical systems.
  arXiv  Detail & Related papers  (2020-03-25T06:00:33Z)
• Adaptive Control and Regret Minimization in Linear Quadratic Gaussian (LQG) Setting [91.43582419264763]
  We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty. LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model.
  arXiv  Detail & Related papers  (2020-03-12T19:56:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.