Multi-Agent Reinforcement Learning for Power Grid Topology Optimization

• URL: http://arxiv.org/abs/2310.02605v1
• Date: Wed, 4 Oct 2023 06:37:43 GMT
• Title: Multi-Agent Reinforcement Learning for Power Grid Topology Optimization
• Authors: Erica van der Sar, Alessandro Zocca, Sandjai Bhulai
• Abstract summary: This paper presents a hierarchical multi-agent reinforcement learning (MARL) framework tailored for expansive action spaces. Experimental results indicate the MARL framework's competitive performance with single-agent RL methods. We also compare different RL algorithms for lower-level agents alongside different policies for higher-order agents.
• Score: 45.74830585715129
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent challenges in operating power networks arise from increasing energy demands and unpredictable renewable sources like wind and solar. While reinforcement learning (RL) shows promise in managing these networks, through topological actions like bus and line switching, efficiently handling large action spaces as networks grow is crucial. This paper presents a hierarchical multi-agent reinforcement learning (MARL) framework tailored for these expansive action spaces, leveraging the power grid's inherent hierarchical nature. Experimental results indicate the MARL framework's competitive performance with single-agent RL methods. We also compare different RL algorithms for lower-level agents alongside different policies for higher-order agents.

Related papers

• State and Action Factorization in Power Grids [47.65236082304256]
  We propose a domain-agnostic algorithm that estimates correlations between state and action components entirely based on data. The algorithm is validated on a power grid benchmark obtained with the Grid2Op simulator.
  arXiv  Detail & Related papers  (2024-09-03T15:00:58Z)
• ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL [80.10358123795946]
  We develop a framework for building multi-turn RL algorithms for fine-tuning large language models. Our framework adopts a hierarchical RL approach and runs two RL algorithms in parallel. Empirically, we find that ArCHer significantly improves efficiency and performance on agent tasks.
  arXiv  Detail & Related papers  (2024-02-29T18:45:56Z)
• Hierarchical Reinforcement Learning for Power Network Topology Control [22.203574989348773]
  Learning in high-dimensional action spaces is a key challenge in applying reinforcement learning to real-world systems. In this paper, we study the possibility of controlling power networks using RL methods.
  arXiv  Detail & Related papers  (2023-11-03T12:33:00Z)
• Attention-based Open RAN Slice Management using Deep Reinforcement Learning [6.177038245239758]
  This paper introduces an innovative attention-based deep RL (ADRL) technique that leverages the O-RAN disaggregated modules and distributed agent cooperation. Simulation results demonstrate significant improvements in network performance compared to other DRL baseline methods.
  arXiv  Detail & Related papers  (2023-06-15T20:37:19Z)
• Managing power grids through topology actions: A comparative study between advanced rule-based and reinforcement learning agents [1.8549313085249322]
  Operation of electricity grids has become increasingly complex due to the current upheaval and the increase in renewable energy production. It has been shown that Reinforcement Learning is an efficient and reliable approach with considerable potential for automatic grid operation. In this article, we analyse the submitted agent from Binbinchen and provide novel strategies to improve the agent, both for the RL and the rule-based approach.
  arXiv  Detail & Related papers  (2023-04-03T07:34:43Z)
• Distributed Energy Management and Demand Response in Smart Grids: A Multi-Agent Deep Reinforcement Learning Framework [53.97223237572147]
  This paper presents a multi-agent Deep Reinforcement Learning (DRL) framework for autonomous control and integration of renewable energy resources into smart power grid systems. In particular, the proposed framework jointly considers demand response (DR) and distributed energy management (DEM) for residential end-users.
  arXiv  Detail & Related papers  (2022-11-29T01:18:58Z)
• Stabilizing Voltage in Power Distribution Networks via Multi-Agent Reinforcement Learning with Transformer [128.19212716007794]
  We propose a Transformer-based Multi-Agent Actor-Critic framework (T-MAAC) to stabilize voltage in power distribution networks. In addition, we adopt a novel auxiliary-task training process tailored to the voltage control task, which improves the sample efficiency.
  arXiv  Detail & Related papers  (2022-06-08T07:48:42Z)
• Multi-Agent Broad Reinforcement Learning for Intelligent Traffic Light Control [21.87935026688773]
  Existing approaches of Multi-Agent System (MAS) are largely based on Multi-Agent Deep Reinforcement Learning (MADRL) We propose a Multi-Agent Broad Reinforcement Learning (MABRL) framework to explore the function of BLS in MAS.
  arXiv  Detail & Related papers  (2022-03-08T14:04:09Z)
• Improving Robustness of Reinforcement Learning for Power System Control with Adversarial Training [71.7750435554693]
  We show that several state-of-the-art RL agents proposed for power system control are vulnerable to adversarial attacks. Specifically, we use an adversary Markov Decision Process to learn an attack policy, and demonstrate the potency of our attack. We propose to use adversarial training to increase the robustness of RL agent against attacks and avoid infeasible operational decisions.
  arXiv  Detail & Related papers  (2021-10-18T00:50:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.