Rethink AI-based Power Grid Control: Diving Into Algorithm Design
- URL: http://arxiv.org/abs/2012.13026v1
- Date: Wed, 23 Dec 2020 23:38:41 GMT
- Title: Rethink AI-based Power Grid Control: Diving Into Algorithm Design
- Authors: Xiren Zhou and Siqi Wang and Ruisheng Diao and Desong Bian and Jiahui
Duan and Di Shi
- Abstract summary: In this paper, we present an in-depth analysis of DRL-based voltage control fromaspects of algorithm selection, state space representation, and reward engineering.
We propose a novel imitation learning-based approachto directly map power grid operating points to effective actions without any interimreinforcement learning process.
- Score: 6.194042945960622
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Recently, deep reinforcement learning (DRL)-based approach has shown
promisein solving complex decision and control problems in power engineering
domain.In this paper, we present an in-depth analysis of DRL-based voltage
control fromaspects of algorithm selection, state space representation, and
reward engineering.To resolve observed issues, we propose a novel imitation
learning-based approachto directly map power grid operating points to effective
actions without any interimreinforcement learning process. The performance
results demonstrate that theproposed approach has strong generalization ability
with much less training time.The agent trained by imitation learning is
effective and robust to solve voltagecontrol problem and outperforms the former
RL agents.
Related papers
- Efficient Deep Reinforcement Learning Requires Regulating Overfitting [91.88004732618381]
We show that high temporal-difference (TD) error on the validation set of transitions is the main culprit that severely affects the performance of deep RL algorithms.
We show that a simple online model selection method that targets the validation TD error is effective across state-based DMC and Gym tasks.
arXiv Detail & Related papers (2023-04-20T17:11:05Z) - MARLIN: Soft Actor-Critic based Reinforcement Learning for Congestion
Control in Real Networks [63.24965775030673]
We propose a novel Reinforcement Learning (RL) approach to design generic Congestion Control (CC) algorithms.
Our solution, MARLIN, uses the Soft Actor-Critic algorithm to maximize both entropy and return.
We trained MARLIN on a real network with varying background traffic patterns to overcome the sim-to-real mismatch.
arXiv Detail & Related papers (2023-02-02T18:27:20Z) - Efficient Learning of Voltage Control Strategies via Model-based Deep
Reinforcement Learning [9.936452412191326]
This article proposes a model-based deep reinforcement learning (DRL) method to design emergency control strategies for short-term voltage stability problems in power systems.
Recent advances show promising results in model-free DRL-based methods for power systems, but model-free methods suffer from poor sample efficiency and training time.
We propose a novel model-based-DRL framework where a deep neural network (DNN)-based dynamic surrogate model is utilized with the policy learning framework.
arXiv Detail & Related papers (2022-12-06T02:50:53Z) - Constrained Reinforcement Learning for Robotics via Scenario-Based
Programming [64.07167316957533]
It is crucial to optimize the performance of DRL-based agents while providing guarantees about their behavior.
This paper presents a novel technique for incorporating domain-expert knowledge into a constrained DRL training loop.
Our experiments demonstrate that using our approach to leverage expert knowledge dramatically improves the safety and the performance of the agent.
arXiv Detail & Related papers (2022-06-20T07:19:38Z) - Jump-Start Reinforcement Learning [68.82380421479675]
We present a meta algorithm that can use offline data, demonstrations, or a pre-existing policy to initialize an RL policy.
In particular, we propose Jump-Start Reinforcement Learning (JSRL), an algorithm that employs two policies to solve tasks.
We show via experiments that JSRL is able to significantly outperform existing imitation and reinforcement learning algorithms.
arXiv Detail & Related papers (2022-04-05T17:25:22Z) - Physics-informed Evolutionary Strategy based Control for Mitigating
Delayed Voltage Recovery [14.44961822756759]
We propose a novel data-driven, real-time power system voltage control method based on the physics-informed guided meta evolutionary strategy (ES)
The main objective is to quickly provide an adaptive control strategy to mitigate the fault-induced delayed voltage recovery (FIDVR) problem.
arXiv Detail & Related papers (2021-11-29T07:12:40Z) - Improving Robustness of Reinforcement Learning for Power System Control
with Adversarial Training [71.7750435554693]
We show that several state-of-the-art RL agents proposed for power system control are vulnerable to adversarial attacks.
Specifically, we use an adversary Markov Decision Process to learn an attack policy, and demonstrate the potency of our attack.
We propose to use adversarial training to increase the robustness of RL agent against attacks and avoid infeasible operational decisions.
arXiv Detail & Related papers (2021-10-18T00:50:34Z) - A Heuristically Assisted Deep Reinforcement Learning Approach for
Network Slice Placement [0.7885276250519428]
We introduce a hybrid placement solution based on Deep Reinforcement Learning (DRL) and a dedicated optimization based on the Power of Two Choices principle.
The proposed Heuristically-Assisted DRL (HA-DRL) allows to accelerate the learning process and gain in resource usage when compared against other state-of-the-art approaches.
arXiv Detail & Related papers (2021-05-14T10:04:17Z) - Consensus Multi-Agent Reinforcement Learning for Volt-VAR Control in
Power Distribution Networks [8.472603460083375]
We propose consensus multi-agent deep reinforcement learning algorithm to solve the VVC problem.
The proposed algorithm allows individual agents to learn a group control policy using local rewards.
Numerical studies on IEEE distribution test feeders show that our proposed algorithm matches the performance of single-agent reinforcement learning benchmark.
arXiv Detail & Related papers (2020-07-06T18:21:47Z) - Information Theoretic Model Predictive Q-Learning [64.74041985237105]
We present a novel theoretical connection between information theoretic MPC and entropy regularized RL.
We develop a Q-learning algorithm that can leverage biased models.
arXiv Detail & Related papers (2019-12-31T00:29:22Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.