Reinforcement Learning for Resilient Power Grids

• URL: http://arxiv.org/abs/2212.04069v1
• Date: Thu, 8 Dec 2022 04:40:14 GMT
• Title: Reinforcement Learning for Resilient Power Grids
• Authors: Zhenting Zhao, Po-Yen Chen, Yucheng Jin
• Abstract summary: Traditional power grid systems have become obsolete under more frequent and extreme natural disasters. Most power grid simulators and RL interfaces do not support simulation of power grid under large-scale blackouts or when the network is divided into sub-networks. In this study, we proposed an updated power grid simulator built on Grid2Op, an existing simulator and RL interface, and experimented on limiting the action and observation spaces of Grid2Op.
• Score: 0.23204178451683263
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Traditional power grid systems have become obsolete under more frequent and extreme natural disasters. Reinforcement learning (RL) has been a promising solution for resilience given its successful history of power grid control. However, most power grid simulators and RL interfaces do not support simulation of power grid under large-scale blackouts or when the network is divided into sub-networks. In this study, we proposed an updated power grid simulator built on Grid2Op, an existing simulator and RL interface, and experimented on limiting the action and observation spaces of Grid2Op. By testing with DDQN and SliceRDQN algorithms, we found that reduced action spaces significantly improve training performance and efficiency. In addition, we investigated a low-rank neural network regularization method for deep Q-learning, one of the most widely used RL algorithms, in this power grid control scenario. As a result, the experiment demonstrated that in the power grid simulation environment, adopting this method will significantly increase the performance of RL agents.

Related papers

• State and Action Factorization in Power Grids [47.65236082304256]
  We propose a domain-agnostic algorithm that estimates correlations between state and action components entirely based on data. The algorithm is validated on a power grid benchmark obtained with the Grid2Op simulator.
  arXiv  Detail & Related papers  (2024-09-03T15:00:58Z)
• Managing power grids through topology actions: A comparative study between advanced rule-based and reinforcement learning agents [1.8549313085249322]
  Operation of electricity grids has become increasingly complex due to the current upheaval and the increase in renewable energy production. It has been shown that Reinforcement Learning is an efficient and reliable approach with considerable potential for automatic grid operation. In this article, we analyse the submitted agent from Binbinchen and provide novel strategies to improve the agent, both for the RL and the rule-based approach.
  arXiv  Detail & Related papers  (2023-04-03T07:34:43Z)
• Reinforcement Learning Based Power Grid Day-Ahead Planning and AI-Assisted Control [0.27998963147546135]
  We introduce a congestion management approach consisting of a redispatching agent and a machine learning-based optimization agent. Compared to a typical redispatching-only agent, it was able to keep a simulated grid in operation longer while at the same time reducing operational cost. The aim of this paper is to bring this promising technology closer to the real world of power grid operation.
  arXiv  Detail & Related papers  (2023-02-15T13:38:40Z)
• Distributed Energy Management and Demand Response in Smart Grids: A Multi-Agent Deep Reinforcement Learning Framework [53.97223237572147]
  This paper presents a multi-agent Deep Reinforcement Learning (DRL) framework for autonomous control and integration of renewable energy resources into smart power grid systems. In particular, the proposed framework jointly considers demand response (DR) and distributed energy management (DEM) for residential end-users.
  arXiv  Detail & Related papers  (2022-11-29T01:18:58Z)
• Stabilizing Voltage in Power Distribution Networks via Multi-Agent Reinforcement Learning with Transformer [128.19212716007794]
  We propose a Transformer-based Multi-Agent Actor-Critic framework (T-MAAC) to stabilize voltage in power distribution networks. In addition, we adopt a novel auxiliary-task training process tailored to the voltage control task, which improves the sample efficiency.
  arXiv  Detail & Related papers  (2022-06-08T07:48:42Z)
• Curriculum Based Reinforcement Learning of Grid Topology Controllers to Prevent Thermal Cascading [0.19116784879310028]
  This paper describes how domain knowledge of power system operators can be integrated into reinforcement learning frameworks. A curriculum-based approach with reward tuning is incorporated into the training procedure by modifying the environment. A parallel training approach on multiple scenarios is employed to avoid biasing the agent to a few scenarios and make it robust to the natural variability in grid operations.
  arXiv  Detail & Related papers  (2021-12-18T20:32:05Z)
• Improving Robustness of Reinforcement Learning for Power System Control with Adversarial Training [71.7750435554693]
  We show that several state-of-the-art RL agents proposed for power system control are vulnerable to adversarial attacks. Specifically, we use an adversary Markov Decision Process to learn an attack policy, and demonstrate the potency of our attack. We propose to use adversarial training to increase the robustness of RL agent against attacks and avoid infeasible operational decisions.
  arXiv  Detail & Related papers  (2021-10-18T00:50:34Z)
• Reinforcement Learning for Datacenter Congestion Control [50.225885814524304]
  Successful congestion control algorithms can dramatically improve latency and overall network throughput. Until today, no such learning-based algorithms have shown practical potential in this domain. We devise an RL-based algorithm with the aim of generalizing to different configurations of real-world datacenter networks. We show that this scheme outperforms alternative popular RL approaches, and generalizes to scenarios that were not seen during training.
  arXiv  Detail & Related papers  (2021-02-18T13:49:28Z)
• Deep Reinforcement Learning for Electric Transmission Voltage Control [0.0]
  A subset of machine learning known as deep reinforcement learning (DRL) has recently shown promise in performing tasks typically performed by humans. This paper applies DRL to the transmission voltage control problem, presents open-source DRL environments for voltage control, and performs experiments at scale with systems up to 500 buses.
  arXiv  Detail & Related papers  (2020-06-11T18:35:25Z)
• Training End-to-End Analog Neural Networks with Equilibrium Propagation [64.0476282000118]
  We introduce a principled method to train end-to-end analog neural networks by gradient descent. We show mathematically that a class of analog neural networks (called nonlinear resistive networks) are energy-based models. Our work can guide the development of a new generation of ultra-fast, compact and low-power neural networks supporting on-chip learning.
  arXiv  Detail & Related papers  (2020-06-02T23:38:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.