Consensus Multi-Agent Reinforcement Learning for Volt-VAR Control in Power Distribution Networks

• URL: http://arxiv.org/abs/2007.02991v1
• Date: Mon, 6 Jul 2020 18:21:47 GMT
• Title: Consensus Multi-Agent Reinforcement Learning for Volt-VAR Control in Power Distribution Networks
• Authors: Yuanqi Gao, Wei Wang, Nanpeng Yu
• Abstract summary: We propose consensus multi-agent deep reinforcement learning algorithm to solve the VVC problem. The proposed algorithm allows individual agents to learn a group control policy using local rewards. Numerical studies on IEEE distribution test feeders show that our proposed algorithm matches the performance of single-agent reinforcement learning benchmark.
• Score: 8.472603460083375
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Volt-VAR control (VVC) is a critical application in active distribution network management system to reduce network losses and improve voltage profile. To remove dependency on inaccurate and incomplete network models and enhance resiliency against communication or controller failure, we propose consensus multi-agent deep reinforcement learning algorithm to solve the VVC problem. The VVC problem is formulated as a networked multi-agent Markov decision process, which is solved using the maximum entropy reinforcement learning framework and a novel communication-efficient consensus strategy. The proposed algorithm allows individual agents to learn a group control policy using local rewards. Numerical studies on IEEE distribution test feeders show that our proposed algorithm matches the performance of single-agent reinforcement learning benchmark. In addition, the proposed algorithm is shown to be communication efficient and resilient.

Related papers

• Augmented Lagrangian-Based Safe Reinforcement Learning Approach for Distribution System Volt/VAR Control [1.1059341532498634]
  This paper formulates the Volt- VAR control problem as a constrained Markov decision process (CMDP) A novel safe off-policy reinforcement learning (RL) approach is proposed in this paper to solve the CMDP. A two-stage strategy is adopted for offline training and online execution, so the accurate distribution system model is no longer needed.
  arXiv  Detail & Related papers  (2024-10-19T19:45:09Z)
• Design Optimization of NOMA Aided Multi-STAR-RIS for Indoor Environments: A Convex Approximation Imitated Reinforcement Learning Approach [51.63921041249406]
  Non-orthogonal multiple access (NOMA) enables multiple users to share the same frequency band, and simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) deploying STAR-RIS indoors presents challenges in interference mitigation, power consumption, and real-time configuration. A novel network architecture utilizing multiple access points (APs), STAR-RISs, and NOMA is proposed for indoor communication.
  arXiv  Detail & Related papers  (2024-06-19T07:17:04Z)
• Decentralized Learning Strategies for Estimation Error Minimization with Graph Neural Networks [94.2860766709971]
  We address the challenge of sampling and remote estimation for autoregressive Markovian processes in a wireless network with statistically-identical agents. Our goal is to minimize time-average estimation error and/or age of information with decentralized scalable sampling and transmission policies.
  arXiv  Detail & Related papers  (2024-04-04T06:24:11Z)
• Distributed-Training-and-Execution Multi-Agent Reinforcement Learning for Power Control in HetNet [48.96004919910818]
  We propose a multi-agent deep reinforcement learning (MADRL) based power control scheme for the HetNet. To promote cooperation among agents, we develop a penalty-based Q learning (PQL) algorithm for MADRL systems. In this way, an agent's policy can be learned by other agents more easily, resulting in a more efficient collaboration process.
  arXiv  Detail & Related papers  (2022-12-15T17:01:56Z)
• State-Augmented Learnable Algorithms for Resource Management in Wireless Networks [124.89036526192268]
  We propose a state-augmented algorithm for solving resource management problems in wireless networks. We show that the proposed algorithm leads to feasible and near-optimal RRM decisions.
  arXiv  Detail & Related papers  (2022-07-05T18:02:54Z)
• Stabilizing Voltage in Power Distribution Networks via Multi-Agent Reinforcement Learning with Transformer [128.19212716007794]
  We propose a Transformer-based Multi-Agent Actor-Critic framework (T-MAAC) to stabilize voltage in power distribution networks. In addition, we adopt a novel auxiliary-task training process tailored to the voltage control task, which improves the sample efficiency.
  arXiv  Detail & Related papers  (2022-06-08T07:48:42Z)
• Optimization for Master-UAV-powered Auxiliary-Aerial-IRS-assisted IoT Networks: An Option-based Multi-agent Hierarchical Deep Reinforcement Learning Approach [56.84948632954274]
  This paper investigates a master unmanned aerial vehicle (MUAV)-powered Internet of Things (IoT) network. We propose using a rechargeable auxiliary UAV (AUAV) equipped with an intelligent reflecting surface (IRS) to enhance the communication signals from the MUAV. Under the proposed model, we investigate the optimal collaboration strategy of these energy-limited UAVs to maximize the accumulated throughput of the IoT network.
  arXiv  Detail & Related papers  (2021-12-20T15:45:28Z)
• A Heuristically Assisted Deep Reinforcement Learning Approach for Network Slice Placement [0.7885276250519428]
  We introduce a hybrid placement solution based on Deep Reinforcement Learning (DRL) and a dedicated optimization based on the Power of Two Choices principle. The proposed Heuristically-Assisted DRL (HA-DRL) allows to accelerate the learning process and gain in resource usage when compared against other state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-05-14T10:04:17Z)
• Rethink AI-based Power Grid Control: Diving Into Algorithm Design [6.194042945960622]
  In this paper, we present an in-depth analysis of DRL-based voltage control fromaspects of algorithm selection, state space representation, and reward engineering. We propose a novel imitation learning-based approachto directly map power grid operating points to effective actions without any interimreinforcement learning process.
  arXiv  Detail & Related papers  (2020-12-23T23:38:41Z)
• Deep Actor-Critic Learning for Distributed Power Control in Wireless Mobile Networks [5.930707872313038]
  Deep reinforcement learning offers a model-free alternative to supervised deep learning and classical optimization. We present a distributively executed continuous power control algorithm with the help of deep actor-critic learning. We integrate the proposed power control algorithm to a time-slotted system where devices are mobile and channel conditions change rapidly.
  arXiv  Detail & Related papers  (2020-09-14T18:29:12Z)
• Distributed Voltage Regulation of Active Distribution System Based on Enhanced Multi-agent Deep Reinforcement Learning [9.7314654861242]
  This paper proposes a data-driven distributed voltage control approach based on the spectrum clustering and the enhanced multi-agent deep reinforcement learning (MADRL) algorithm. The proposed method can significantly reduce the requirements of communications and knowledge of system parameters. It also effectively deals with uncertainties and can provide online coordinated control based on the latest local information.
  arXiv  Detail & Related papers  (2020-05-31T15:48:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.