WFCRL: A Multi-Agent Reinforcement Learning Benchmark for Wind Farm Control

• URL: http://arxiv.org/abs/2501.13592v1
• Date: Thu, 23 Jan 2025 12:01:17 GMT
• Title: WFCRL: A Multi-Agent Reinforcement Learning Benchmark for Wind Farm Control
• Authors: Claire Bizon Monroc, Ana Bušić, Donatien Dubuc, Jiamin Zhu,
• Abstract summary: We introduce WFCRL (Wind Farm Control with Reinforcement Learning), the first open suite of multi-agent reinforcement learning environments for the wind farm control problem. Each turbine is an agent and can learn to adjust its yaw, pitch or torque to maximize the common objective. For each simulator, $10$ wind layouts are provided, including $5$ real wind farms.
• Score: 0.9374652839580183
• License:
• Abstract: The wind farm control problem is challenging, since conventional model-based control strategies require tractable models of complex aerodynamical interactions between the turbines and suffer from the curse of dimension when the number of turbines increases. Recently, model-free and multi-agent reinforcement learning approaches have been used to address this challenge. In this article, we introduce WFCRL (Wind Farm Control with Reinforcement Learning), the first open suite of multi-agent reinforcement learning environments for the wind farm control problem. WFCRL frames a cooperative Multi-Agent Reinforcement Learning (MARL) problem: each turbine is an agent and can learn to adjust its yaw, pitch or torque to maximize the common objective (e.g. the total power production of the farm). WFCRL also offers turbine load observations that will allow to optimize the farm performance while limiting turbine structural damages. Interfaces with two state-of-the-art farm simulators are implemented in WFCRL: a static simulator (FLORIS) and a dynamic simulator (FAST.Farm). For each simulator, $10$ wind layouts are provided, including $5$ real wind farms. Two state-of-the-art online MARL algorithms are implemented to illustrate the scaling challenges. As learning online on FAST.Farm is highly time-consuming, WFCRL offers the possibility of designing transfer learning strategies from FLORIS to FAST.Farm.

Related papers

• Wind turbine condition monitoring based on intra- and inter-farm federated learning [0.0]
  Many AI applications in wind energy may benefit from using operational data not only from individual wind turbines but from multiple turbines and multiple wind farms. Federated learning has emerged as a privacy-preserving distributed machine learning approach in this context. We investigate various federated learning strategies, including collaboration across different wind farms and turbine models, as well as collaboration restricted to the same wind farm and turbine model.
  arXiv  Detail & Related papers  (2024-09-05T16:25:30Z)
• Function Approximation for Reinforcement Learning Controller for Energy from Spread Waves [69.9104427437916]
  Multi-generator Wave Energy Converters (WEC) must handle multiple simultaneous waves coming from different directions called spread waves. These complex devices need controllers with multiple objectives of energy capture efficiency, reduction of structural stress to limit maintenance, and proactive protection against high waves. In this paper, we explore different function approximations for the policy and critic networks in modeling the sequential nature of the system dynamics.
  arXiv  Detail & Related papers  (2024-04-17T02:04:10Z)
• Reinforcement learning to maximise wind turbine energy generation [0.8437187555622164]
  We propose a reinforcement learning strategy to control wind turbine energy generation by actively changing the rotor speed, the rotor yaw angle and the blade pitch angle. A double deep Q-learning with a prioritized experience replay agent is coupled with a blade element momentum model and is trained to allow control for changing winds. The agent is trained to decide the best control (speed, yaw, pitch) for simple steady winds and is subsequently challenged with real dynamic turbulent winds, showing good performance.
  arXiv  Detail & Related papers  (2024-02-17T21:35:13Z)
• Collective Large-scale Wind Farm Multivariate Power Output Control Based on Hierarchical Communication Multi-Agent Proximal Policy Optimization [5.062455071500403]
  Wind power is becoming an increasingly important source of renewable energy worldwide. Wind farm power control faces significant challenges due to the high system complexity inherent in these farms. A novel communication-based multi-agent deep reinforcement learning large-scale wind farm multivariate control is proposed to handle this challenge.
  arXiv  Detail & Related papers  (2023-05-17T12:26:08Z)
• On Transforming Reinforcement Learning by Transformer: The Development Trajectory [97.79247023389445]
  Transformer, originally devised for natural language processing, has also attested significant success in computer vision. We group existing developments in two categories: architecture enhancement and trajectory optimization. We examine the main applications of TRL in robotic manipulation, text-based games, navigation and autonomous driving.
  arXiv  Detail & Related papers  (2022-12-29T03:15:59Z)
• Modeling Wind Turbine Performance and Wake Interactions with Machine Learning [0.0]
  Different machine learning (ML) models are trained on SCADA and meteorological data collected at an onshore wind farm. ML methods for data quality control and pre-processing are applied to the data set under investigation. A hybrid model is found to achieve high accuracy for modeling wind turbine power capture.
  arXiv  Detail & Related papers  (2022-12-02T23:07:05Z)
• Skip Training for Multi-Agent Reinforcement Learning Controller for Industrial Wave Energy Converters [94.84709449845352]
  Recent Wave Energy Converters (WEC) are equipped with multiple legs and generators to maximize energy generation. Traditional controllers have shown limitations to capture complex wave patterns and the controllers must efficiently maximize the energy capture. This paper introduces a Multi-Agent Reinforcement Learning controller (MARL), which outperforms the traditionally used spring damper controller.
  arXiv  Detail & Related papers  (2022-09-13T00:20:31Z)
• Neural-Fly Enables Rapid Learning for Agile Flight in Strong Winds [96.74836678572582]
  We present a learning-based approach that allows rapid online adaptation by incorporating pretrained representations through deep learning. Neural-Fly achieves precise flight control with substantially smaller tracking error than state-of-the-art nonlinear and adaptive controllers.
  arXiv  Detail & Related papers  (2022-05-13T21:55:28Z)
• Efficient Transformers in Reinforcement Learning using Actor-Learner Distillation [91.05073136215886]
  "Actor-Learner Distillation" transfers learning progress from a large capacity learner model to a small capacity actor model. We demonstrate in several challenging memory environments that using Actor-Learner Distillation recovers the clear sample-efficiency gains of the transformer learner model.
  arXiv  Detail & Related papers  (2021-04-04T17:56:34Z)
• UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformers [108.92194081987967]
  We make the first attempt to explore a universal multi-agent reinforcement learning pipeline, designing one single architecture to fit tasks. Unlike previous RNN-based models, we utilize a transformer-based model to generate a flexible policy. The proposed model, named as Universal Policy Decoupling Transformer (UPDeT), further relaxes the action restriction and makes the multi-agent task's decision process more explainable.
  arXiv  Detail & Related papers  (2021-01-20T07:24:24Z)
• Scalable Optimization for Wind Farm Control using Coordination Graphs [5.56699571220921]
  A wind farm controller is required to match the farm's power production with a power demand imposed by the grid operator. This is a non-trivial optimization problem, as complex dependencies exist between the wind turbines. We propose a new learning method for wind farm control that leverages the sparse wind farm structure to factorize the optimization problem.
  arXiv  Detail & Related papers  (2021-01-19T20:12:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.