Distributed Voltage Regulation of Active Distribution System Based on Enhanced Multi-agent Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2006.00546v1
• Date: Sun, 31 May 2020 15:48:27 GMT
• Title: Distributed Voltage Regulation of Active Distribution System Based on Enhanced Multi-agent Deep Reinforcement Learning
• Authors: Di Cao, Junbo Zhao, Weihao Hu, Fei Ding, Qi Huang, Zhe Chen
• Abstract summary: This paper proposes a data-driven distributed voltage control approach based on the spectrum clustering and the enhanced multi-agent deep reinforcement learning (MADRL) algorithm. The proposed method can significantly reduce the requirements of communications and knowledge of system parameters. It also effectively deals with uncertainties and can provide online coordinated control based on the latest local information.
• Score: 9.7314654861242
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper proposes a data-driven distributed voltage control approach based on the spectrum clustering and the enhanced multi-agent deep reinforcement learning (MADRL) algorithm. Via the unsupervised clustering, the whole distribution system can be decomposed into several sub-networks according to the voltage and reactive power sensitivity. Then, the distributed control problem of each sub-network is modeled as Markov games and solved by the enhanced MADRL algorithm, where each sub-network is modeled as an adaptive agent. Deep neural networks are used in each agent to approximate the policy function and the action value function. All agents are centrally trained to learn the optimal coordinated voltage regulation strategy while executed in a distributed manner to make decisions based on only local information. The proposed method can significantly reduce the requirements of communications and knowledge of system parameters. It also effectively deals with uncertainties and can provide online coordinated control based on the latest local information. Comparison results with other existing model-based and data-driven methods on IEEE 33-bus and 123-bus systems demonstrate the effectiveness and benefits of the proposed approach.

Related papers

• Scalable spectral representations for network multiagent control [53.631272539560435]
  A popular model for multi-agent control, Network Markov Decision Processes (MDPs) pose a significant challenge to efficient learning. We first derive scalable spectral local representations for network MDPs, which induces a network linear subspace for the local $Q$-function of each agent. We design a scalable algorithmic framework for continuous state-action network MDPs, and provide end-to-end guarantees for the convergence of our algorithm.
  arXiv  Detail & Related papers  (2024-10-22T17:45:45Z)
• Augmented Lagrangian-Based Safe Reinforcement Learning Approach for Distribution System Volt/VAR Control [1.1059341532498634]
  This paper formulates the Volt- VAR control problem as a constrained Markov decision process (CMDP) A novel safe off-policy reinforcement learning (RL) approach is proposed in this paper to solve the CMDP. A two-stage strategy is adopted for offline training and online execution, so the accurate distribution system model is no longer needed.
  arXiv  Detail & Related papers  (2024-10-19T19:45:09Z)
• Decentralized Learning Strategies for Estimation Error Minimization with Graph Neural Networks [94.2860766709971]
  We address the challenge of sampling and remote estimation for autoregressive Markovian processes in a wireless network with statistically-identical agents. Our goal is to minimize time-average estimation error and/or age of information with decentralized scalable sampling and transmission policies.
  arXiv  Detail & Related papers  (2024-04-04T06:24:11Z)
• Distributed-Training-and-Execution Multi-Agent Reinforcement Learning for Power Control in HetNet [48.96004919910818]
  We propose a multi-agent deep reinforcement learning (MADRL) based power control scheme for the HetNet. To promote cooperation among agents, we develop a penalty-based Q learning (PQL) algorithm for MADRL systems. In this way, an agent's policy can be learned by other agents more easily, resulting in a more efficient collaboration process.
  arXiv  Detail & Related papers  (2022-12-15T17:01:56Z)
• Adaptive Stochastic ADMM for Decentralized Reinforcement Learning in Edge Industrial IoT [106.83952081124195]
  Reinforcement learning (RL) has been widely investigated and shown to be a promising solution for decision-making and optimal control processes. We propose an adaptive ADMM (asI-ADMM) algorithm and apply it to decentralized RL with edge-computing-empowered IIoT networks. Experiment results show that our proposed algorithms outperform the state of the art in terms of communication costs and scalability, and can well adapt to complex IoT environments.
  arXiv  Detail & Related papers  (2021-06-30T16:49:07Z)
• Learning Autonomy in Management of Wireless Random Networks [102.02142856863563]
  This paper presents a machine learning strategy that tackles a distributed optimization task in a wireless network with an arbitrary number of randomly interconnected nodes. We develop a flexible deep neural network formalism termed distributed message-passing neural network (DMPNN) with forward and backward computations independent of the network topology.
  arXiv  Detail & Related papers  (2021-06-15T09:03:28Z)
• Distributed Algorithms for Linearly-Solvable Optimal Control in Networked Multi-Agent Systems [15.782670973813774]
  A distributed framework is proposed to partition the optimal control problem of a networked MAS into several local optimal control problems. For discrete-time systems, the joint Bellman equation of each subsystem is transformed into a system of linear equations. For continuous-time systems, the joint optimality equation of each subsystem is converted into a linear partial differential equation.
  arXiv  Detail & Related papers  (2021-02-18T01:31:17Z)
• Scalable Voltage Control using Structure-Driven Hierarchical Deep Reinforcement Learning [0.0]
  This paper presents a novel hierarchical deep reinforcement learning (DRL) based design for the voltage control of power grids. We exploit the area-wise division structure of the power system to propose a hierarchical DRL design that can be scaled to the larger grid models. We train area-wise decentralized RL agents to compute lower-level policies for the individual areas, and concurrently train a higher-level DRL agent that uses the updates of the lower-level policies to efficiently coordinate the control actions taken by the lower-level agents.
  arXiv  Detail & Related papers  (2021-01-29T21:30:59Z)
• Lyapunov-Based Reinforcement Learning for Decentralized Multi-Agent Control [3.3788926259119645]
  In decentralized multi-agent control, systems are complex with unknown or highly uncertain dynamics. Deep reinforcement learning (DRL) is promising to learn the controller/policy from data without the knowing system dynamics. Existing multi-agent reinforcement learning (MARL) algorithms cannot ensure the closed-loop stability of a multi-agent system. We propose a new MARL algorithm for decentralized multi-agent control with a stability guarantee.
  arXiv  Detail & Related papers  (2020-09-20T06:11:42Z)
• Consensus Multi-Agent Reinforcement Learning for Volt-VAR Control in Power Distribution Networks [8.472603460083375]
  We propose consensus multi-agent deep reinforcement learning algorithm to solve the VVC problem. The proposed algorithm allows individual agents to learn a group control policy using local rewards. Numerical studies on IEEE distribution test feeders show that our proposed algorithm matches the performance of single-agent reinforcement learning benchmark.
  arXiv  Detail & Related papers  (2020-07-06T18:21:47Z)
• Decentralized MCTS via Learned Teammate Models [89.24858306636816]
  We present a trainable online decentralized planning algorithm based on decentralized Monte Carlo Tree Search. We show that deep learning and convolutional neural networks can be employed to produce accurate policy approximators.
  arXiv  Detail & Related papers  (2020-03-19T13:10:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.