Hierarchical RL-MPC Control for Dynamic Wake Steering in Wind Farms

• URL: http://arxiv.org/abs/2604.22797v1
• Date: Mon, 13 Apr 2026 12:57:38 GMT
• Title: Hierarchical RL-MPC Control for Dynamic Wake Steering in Wind Farms
• Abstract summary: This paper presents a hierarchical framework that combines reinforcement learning with model predictive control, where an RL agent learns compensatory state estimates for an MPC controller, rather than directly controlling turbines.<n>Compared to direct RL control, the hybrid architecture maintains superior safety characteristics during training while achieving comparable performance with more stable control actions.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Wind farm wake steering optimization is challenging due to complex flow physics and changing conditions. This paper presents a hierarchical framework that combines reinforcement learning with model predictive control, where an RL agent learns compensatory state estimates for an MPC controller, rather than directly controlling turbines. Evaluated on a three-turbine case, the approach achieves a 23\% power gain over the baseline control and surpasses the idealized MPC with perfect state knowledge. Compared to direct RL control, the hybrid architecture maintains superior safety characteristics during training while achieving comparable performance with more stable control actions.

Related papers

• High-Precision Formation Control for Heterogeneous Multi-Robot Systems via Hierarchical Hybrid Physics-Informed Deep Reinforcement Learning [10.865402396881416]
  This paper proposes a hierarchical hybrid physics-informed deep reinforcement learning (HHy-PIDRL) framework.<n>It aims to realize high-precision, highly responsive formation control for heterogeneous multi-robot systems.<n> Experimental results demonstrate that, the success rate of both the upper-layer autonomous navigation policy network and the HM-DRL based formation control policy networks reach 100%.
  arXiv  Detail & Related papers  (2026-07-03T17:38:17Z)
• Deep Reinforcement Learning for Spacecraft Attitude Control During Atmospheric Re-Entry [37.22137915015052]
  We explore reinforcement learning for attitude control in spacecraft re-entry.<n>We formalize the application in the RL framework to apply continuous, off-policy RL.<n>We show that hybrid controllers are able to track the angle of attack better and are more robust under variations of mass, inertia tensor, and flap actuator bandwidth.
  arXiv  Detail & Related papers  (2026-06-30T08:06:07Z)
• Beyond Conservative Automated Driving in Multi-Agent Scenarios via Coupled Model Predictive Control and Deep Reinforcement Learning [7.804905867413516]
  Model Predictive Control (MPC) offers structured constraint handling through optimization.<n>Deep Reinforcement Learning (RL) learns adaptive behaviors from experience but often struggles with safety assurance.<n>We present an integrated MPC-RL framework to improve navigation performance in multi-agent scenarios.
  arXiv  Detail & Related papers  (2026-04-15T13:58:38Z)
• Physics-informed Neural-operator Predictive Control for Drag Reduction in Turbulent Flows [109.99020160824553]
  We propose an efficient deep reinforcement learning framework for modeling and control of turbulent flows.<n>It is model-based RL for predictive control (PC), where both the policy and the observer models for turbulence control are learned jointly.<n>We find that PINO-PC achieves a drag reduction of 39.0% under a bulk-velocity Reynolds number of 15,000, outperforming previous fluid control methods by more than 32%.
  arXiv  Detail & Related papers  (2025-10-03T00:18:26Z)
• Improving a Proportional Integral Controller with Reinforcement Learning on a Throttle Valve Benchmark [2.8322124733515666]
  This paper presents a learning-based control strategy for non-linear throttle valves with an asymmetric controller. We exploit the recent advances in Reinforcement Learning with Guides to improve the closed-loop behavior by learning from the additional interactions with the valve. In all the experimental test cases, the resulting agent has a better sample efficiency than traditional RL agents and outperforms the PI controller.
  arXiv  Detail & Related papers  (2024-02-21T09:40:26Z)
• A Reinforcement Learning Approach for Robust Supervisory Control of UAVs Under Disturbances [1.8799681615947088]
  We present an approach to supervisory reinforcement learning control for unmanned aerial vehicles (UAVs) We formulate a supervisory control architecture that interleaves with extant embedded control and demonstrates robustness to environmental disturbances in the form of adverse wind conditions.
  arXiv  Detail & Related papers  (2023-05-21T19:00:06Z)
• Skip Training for Multi-Agent Reinforcement Learning Controller for Industrial Wave Energy Converters [94.84709449845352]
  Recent Wave Energy Converters (WEC) are equipped with multiple legs and generators to maximize energy generation. Traditional controllers have shown limitations to capture complex wave patterns and the controllers must efficiently maximize the energy capture. This paper introduces a Multi-Agent Reinforcement Learning controller (MARL), which outperforms the traditionally used spring damper controller.
  arXiv  Detail & Related papers  (2022-09-13T00:20:31Z)
• Comparative analysis of machine learning methods for active flow control [60.53767050487434]
  Genetic Programming (GP) and Reinforcement Learning (RL) are gaining popularity in flow control. This work presents a comparative analysis of the two, bench-marking some of their most representative algorithms against global optimization techniques.
  arXiv  Detail & Related papers  (2022-02-23T18:11:19Z)
• Policy Search for Model Predictive Control with Application to Agile Drone Flight [56.24908013905407]
  We propose a policy-search-for-model-predictive-control framework for MPC. Specifically, we formulate the MPC as a parameterized controller, where the hard-to-optimize decision variables are represented as high-level policies. Experiments show that our controller achieves robust and real-time control performance in both simulation and the real world.
  arXiv  Detail & Related papers  (2021-12-07T17:39:24Z)
• Scalable Voltage Control using Structure-Driven Hierarchical Deep Reinforcement Learning [0.0]
  This paper presents a novel hierarchical deep reinforcement learning (DRL) based design for the voltage control of power grids. We exploit the area-wise division structure of the power system to propose a hierarchical DRL design that can be scaled to the larger grid models. We train area-wise decentralized RL agents to compute lower-level policies for the individual areas, and concurrently train a higher-level DRL agent that uses the updates of the lower-level policies to efficiently coordinate the control actions taken by the lower-level agents.
  arXiv  Detail & Related papers  (2021-01-29T21:30:59Z)
• Regularizing Action Policies for Smooth Control with Reinforcement Learning [47.312768123967025]
  Conditioning for Action Policy Smoothness (CAPS) is an effective yet intuitive regularization on action policies. CAPS offers consistent improvement in the smoothness of the learned state-to-action mappings of neural network controllers. Tested on a real system, improvements in controller smoothness on a quadrotor drone resulted in an almost 80% reduction in power consumption.
  arXiv  Detail & Related papers  (2020-12-11T21:35:24Z)
• Learning a Contact-Adaptive Controller for Robust, Efficient Legged Locomotion [95.1825179206694]
  We present a framework that synthesizes robust controllers for a quadruped robot. A high-level controller learns to choose from a set of primitives in response to changes in the environment. A low-level controller that utilizes an established control method to robustly execute the primitives.
  arXiv  Detail & Related papers  (2020-09-21T16:49:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.