Optimizing Power Grid Topologies with Reinforcement Learning: A Survey of Methods and Challenges

• URL: http://arxiv.org/abs/2504.08210v1
• Date: Fri, 11 Apr 2025 02:27:30 GMT
• Title: Optimizing Power Grid Topologies with Reinforcement Learning: A Survey of Methods and Challenges
• Authors: Erica van der Sar, Alessandro Zocca, Sandjai Bhulai,
• Abstract summary: Reinforcement Learning (RL) has emerged as a promising approach to power network control (PNC)<n>This survey provides a comprehensive overview of RL applications for power grid topology optimization.
• Score: 42.642008092347986
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Power grid operation is becoming increasingly complex due to the rising integration of renewable energy sources and the need for more adaptive control strategies. Reinforcement Learning (RL) has emerged as a promising approach to power network control (PNC), offering the potential to enhance decision-making in dynamic and uncertain environments. The Learning To Run a Power Network (L2RPN) competitions have played a key role in accelerating research by providing standardized benchmarks and problem formulations, leading to rapid advancements in RL-based methods. This survey provides a comprehensive and structured overview of RL applications for power grid topology optimization, categorizing existing techniques, highlighting key design choices, and identifying gaps in current research. Additionally, we present a comparative numerical study evaluating the impact of commonly applied RL-based methods, offering insights into their practical effectiveness. By consolidating existing research and outlining open challenges, this survey aims to provide a foundation for future advancements in RL-driven power grid optimization.

Related papers

• A Comprehensive Survey of Reinforcement Learning: From Algorithms to Practical Challenges [2.2448567386846916]
  Reinforcement Learning (RL) has emerged as a powerful paradigm in Artificial Intelligence (AI)<n>This paper presents a comprehensive survey of RL, meticulously analyzing a wide range of algorithms.<n>We offer practical insights into the selection and implementation of RL algorithms, addressing common challenges like convergence, stability, and the exploration-exploitation dilemma.
  arXiv  Detail & Related papers  (2024-11-28T03:53:14Z)
• Advanced Intelligent Optimization Algorithms for Multi-Objective Optimal Power Flow in Future Power Systems: A Review [1.450405446885067]
  Review explores the application of intelligent optimization algorithms to Multi-Objective Optimal Power Flow (MOPF) It delves into the challenges posed by the integration of renewables, smart grids, and increasing energy demands. Findings suggest that algorithm selection is contingent on the specific MOPF problem at hand, and hybrid approaches offer significant promise.
  arXiv  Detail & Related papers  (2024-04-14T09:44:08Z)
• Bridging Evolutionary Algorithms and Reinforcement Learning: A Comprehensive Survey on Hybrid Algorithms [50.91348344666895]
  Evolutionary Reinforcement Learning (ERL) integrates Evolutionary Algorithms (EAs) and Reinforcement Learning (RL) for optimization. This survey offers a comprehensive overview of the diverse research branches in ERL.
  arXiv  Detail & Related papers  (2024-01-22T14:06:37Z)
• Deep Reinforcement Learning for Community Battery Scheduling under Uncertainties of Load, PV Generation, and Energy Prices [5.694872363688119]
  This paper presents a deep reinforcement learning (RL) strategy to schedule a community battery system in the presence of uncertainties. We position the community battery to play a versatile role, in integrating local PV energy, reducing peak load, and exploiting energy price fluctuations for arbitrage.
  arXiv  Detail & Related papers  (2023-12-04T13:45:17Z)
• Reinforcement Learning-assisted Evolutionary Algorithm: A Survey and Research Opportunities [63.258517066104446]
  Reinforcement learning integrated as a component in the evolutionary algorithm has demonstrated superior performance in recent years. We discuss the RL-EA integration method, the RL-assisted strategy adopted by RL-EA, and its applications according to the existing literature. In the applications of RL-EA section, we also demonstrate the excellent performance of RL-EA on several benchmarks and a range of public datasets.
  arXiv  Detail & Related papers  (2023-08-25T15:06:05Z)
• Ensemble Reinforcement Learning: A Survey [43.17635633600716]
  Reinforcement Learning (RL) has emerged as a highly effective technique for addressing various scientific and applied problems. In response, ensemble reinforcement learning (ERL), a promising approach that combines the benefits of both RL and ensemble learning (EL), has gained widespread popularity. ERL leverages multiple models or training algorithms to comprehensively explore the problem space and possesses strong generalization capabilities.
  arXiv  Detail & Related papers  (2023-03-05T09:26:44Z)
• MARLIN: Soft Actor-Critic based Reinforcement Learning for Congestion Control in Real Networks [63.24965775030673]
  We propose a novel Reinforcement Learning (RL) approach to design generic Congestion Control (CC) algorithms. Our solution, MARLIN, uses the Soft Actor-Critic algorithm to maximize both entropy and return. We trained MARLIN on a real network with varying background traffic patterns to overcome the sim-to-real mismatch.
  arXiv  Detail & Related papers  (2023-02-02T18:27:20Z)
• Jump-Start Reinforcement Learning [68.82380421479675]
  We present a meta algorithm that can use offline data, demonstrations, or a pre-existing policy to initialize an RL policy. In particular, we propose Jump-Start Reinforcement Learning (JSRL), an algorithm that employs two policies to solve tasks. We show via experiments that JSRL is able to significantly outperform existing imitation and reinforcement learning algorithms.
  arXiv  Detail & Related papers  (2022-04-05T17:25:22Z)
• Improving Robustness of Reinforcement Learning for Power System Control with Adversarial Training [71.7750435554693]
  We show that several state-of-the-art RL agents proposed for power system control are vulnerable to adversarial attacks. Specifically, we use an adversary Markov Decision Process to learn an attack policy, and demonstrate the potency of our attack. We propose to use adversarial training to increase the robustness of RL agent against attacks and avoid infeasible operational decisions.
  arXiv  Detail & Related papers  (2021-10-18T00:50:34Z)
• A Heuristically Assisted Deep Reinforcement Learning Approach for Network Slice Placement [0.7885276250519428]
  We introduce a hybrid placement solution based on Deep Reinforcement Learning (DRL) and a dedicated optimization based on the Power of Two Choices principle. The proposed Heuristically-Assisted DRL (HA-DRL) allows to accelerate the learning process and gain in resource usage when compared against other state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-05-14T10:04:17Z)
• Rethink AI-based Power Grid Control: Diving Into Algorithm Design [6.194042945960622]
  In this paper, we present an in-depth analysis of DRL-based voltage control fromaspects of algorithm selection, state space representation, and reward engineering. We propose a novel imitation learning-based approachto directly map power grid operating points to effective actions without any interimreinforcement learning process.
  arXiv  Detail & Related papers  (2020-12-23T23:38:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.