Stabilizing Voltage in Power Distribution Networks via Multi-Agent Reinforcement Learning with Transformer

• URL: http://arxiv.org/abs/2206.03721v1
• Date: Wed, 8 Jun 2022 07:48:42 GMT
• Title: Stabilizing Voltage in Power Distribution Networks via Multi-Agent Reinforcement Learning with Transformer
• Authors: Minrui Wang, Mingxiao Feng, Wengang Zhou, Houqiang Li
• Abstract summary: We propose a Transformer-based Multi-Agent Actor-Critic framework (T-MAAC) to stabilize voltage in power distribution networks. In addition, we adopt a novel auxiliary-task training process tailored to the voltage control task, which improves the sample efficiency.
• Score: 128.19212716007794
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The increased integration of renewable energy poses a slew of technical challenges for the operation of power distribution networks. Among them, voltage fluctuations caused by the instability of renewable energy are receiving increasing attention. Utilizing MARL algorithms to coordinate multiple control units in the grid, which is able to handle rapid changes of power systems, has been widely studied in active voltage control task recently. However, existing approaches based on MARL ignore the unique nature of the grid and achieve limited performance. In this paper, we introduce the transformer architecture to extract representations adapting to power network problems and propose a Transformer-based Multi-Agent Actor-Critic framework (T-MAAC) to stabilize voltage in power distribution networks. In addition, we adopt a novel auxiliary-task training process tailored to the voltage control task, which improves the sample efficiency and facilitating the representation learning of the transformer-based model. We couple T-MAAC with different multi-agent actor-critic algorithms, and the consistent improvements on the active voltage control task demonstrate the effectiveness of the proposed method.

Related papers

• Fair Reinforcement Learning Algorithm for PV Active Control in LV Distribution Networks [1.75493501156941]
  The adoption of distributed energy resources has presented new and complex challenges for power network control. With the significant energy production from PV panels, voltage issues in the network have become a problem. Reducing the active power output of PV panels can be perceived as unfair to some customers, discouraging future installations. A reinforcement learning technique is proposed to address voltage issues in a distribution network, while considering fairness in active power curtailment.
  arXiv  Detail & Related papers  (2024-09-09T10:51:08Z)
• Transmission Interface Power Flow Adjustment: A Deep Reinforcement Learning Approach based on Multi-task Attribution Map [33.929818014940054]
  We introduce a novel data-driven deep reinforcement learning (DRL) approach to handle multiple power flow adjustment tasks jointly. At the heart of the proposed method is a multi-task attribution map (MAM), which enables the DRL agent to explicitly attribute each transmission interface task to different power system nodes. Based on this MAM, the agent can further provide effective strategies to solve the multi-task adjustment problem with a near-optimal operation cost.
  arXiv  Detail & Related papers  (2024-05-24T08:20:53Z)
• Function Approximation for Reinforcement Learning Controller for Energy from Spread Waves [69.9104427437916]
  Multi-generator Wave Energy Converters (WEC) must handle multiple simultaneous waves coming from different directions called spread waves. These complex devices need controllers with multiple objectives of energy capture efficiency, reduction of structural stress to limit maintenance, and proactive protection against high waves. In this paper, we explore different function approximations for the policy and critic networks in modeling the sequential nature of the system dynamics.
  arXiv  Detail & Related papers  (2024-04-17T02:04:10Z)
• Distributed Energy Management and Demand Response in Smart Grids: A Multi-Agent Deep Reinforcement Learning Framework [53.97223237572147]
  This paper presents a multi-agent Deep Reinforcement Learning (DRL) framework for autonomous control and integration of renewable energy resources into smart power grid systems. In particular, the proposed framework jointly considers demand response (DR) and distributed energy management (DEM) for residential end-users.
  arXiv  Detail & Related papers  (2022-11-29T01:18:58Z)
• Over-the-Air Federated Multi-Task Learning via Model Sparsification and Turbo Compressed Sensing [48.19771515107681]
  We propose an over-the-air FMTL framework, where multiple learning tasks deployed on edge devices share a non-orthogonal fading channel under the coordination of an edge server. In OA-FMTL, the local updates of edge devices are sparsified, compressed, and then sent over the uplink channel in a superimposed fashion. We analyze the performance of the proposed OA-FMTL framework together with the M-Turbo-CS algorithm.
  arXiv  Detail & Related papers  (2022-05-08T08:03:52Z)
• Towards Lightweight Transformer via Group-wise Transformation for Vision-and-Language Tasks [126.33843752332139]
  We introduce Group-wise Transformation towards a universal yet lightweight Transformer for vision-and-language tasks, termed as LW-Transformer. We apply LW-Transformer to a set of Transformer-based networks, and quantitatively measure them on three vision-and-language tasks and six benchmark datasets. Experimental results show that while saving a large number of parameters and computations, LW-Transformer achieves very competitive performance against the original Transformer networks for vision-and-language tasks.
  arXiv  Detail & Related papers  (2022-04-16T11:30:26Z)
• Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networks [2.992389186393994]
  The emerging trend of decarbonisation is placing excessive stress on power distribution networks. Active voltage control is seen as a promising solution to relieve power congestion and improve voltage quality without extra hardware investment. This paper formulates the active voltage control problem in the framework of Dec-POMDP and establishes an open-source environment.
  arXiv  Detail & Related papers  (2021-10-27T09:31:22Z)
• An Intelligent Control Strategy for buck DC-DC Converter via Deep Reinforcement Learning [1.4502611532302039]
  An innovative intelligent control strategy for buck DC-DC converter with constant power loads (CPLs) is constructed for the first time. A Markov Decision Process (MDP) model and the deep Q network (DQN) algorithm are defined for DC-DC converter. A model-free based deep reinforcement learning (DRL) control strategy is appropriately designed to adjust the agent-environment interaction.
  arXiv  Detail & Related papers  (2020-08-11T06:38:53Z)
• Consensus Multi-Agent Reinforcement Learning for Volt-VAR Control in Power Distribution Networks [8.472603460083375]
  We propose consensus multi-agent deep reinforcement learning algorithm to solve the VVC problem. The proposed algorithm allows individual agents to learn a group control policy using local rewards. Numerical studies on IEEE distribution test feeders show that our proposed algorithm matches the performance of single-agent reinforcement learning benchmark.
  arXiv  Detail & Related papers  (2020-07-06T18:21:47Z)
• Risk-Aware Energy Scheduling for Edge Computing with Microgrid: A Multi-Agent Deep Reinforcement Learning Approach [82.6692222294594]
  We study a risk-aware energy scheduling problem for a microgrid-powered MEC network. We derive the solution by applying a multi-agent deep reinforcement learning (MADRL)-based advantage actor-critic (A3C) algorithm with shared neural networks.
  arXiv  Detail & Related papers  (2020-02-21T02:14:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.