Imitation Learning for Intra-Day Power Grid Operation through Topology Actions

• URL: http://arxiv.org/abs/2407.19865v2
• Date: Sun, 18 Aug 2024 09:55:39 GMT
• Title: Imitation Learning for Intra-Day Power Grid Operation through Topology Actions
• Authors: Matthijs de Jong, Jan Viebahn, Yuliya Shapovalova,
• Abstract summary: We study the performance of imitation learning for day-ahead power grid operation through topology actions. We train a fully-connected neural network (FCNN) on expert state-action pairs and evaluate it in two ways. As a power system agent, the FCNN performs only slightly worse than expert agents.
• Score: 0.24578723416255752
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Power grid operation is becoming increasingly complex due to the increase in generation of renewable energy. The recent series of Learning To Run a Power Network (L2RPN) competitions have encouraged the use of artificial agents to assist human dispatchers in operating power grids. In this paper we study the performance of imitation learning for day-ahead power grid operation through topology actions. In particular, we consider two rule-based expert agents: a greedy agent and a N-1 agent. While the latter is more computationally expensive since it takes N-1 safety considerations into account, it exhibits a much higher operational performance. We train a fully-connected neural network (FCNN) on expert state-action pairs and evaluate it in two ways. First, we find that classification accuracy is limited despite extensive hyperparameter tuning, due to class imbalance and class overlap. Second, as a power system agent, the FCNN performs only slightly worse than expert agents. Furthermore, hybrid agents, which incorporate minimal additional simulations, match expert agents' performance with significantly lower computational cost. Consequently, imitation learning shows promise for developing fast, high-performing power grid agents, motivating its further exploration in future L2RPN studies.

Related papers

• Robust Defense Against Extreme Grid Events Using Dual-Policy Reinforcement Learning Agents [0.0]
  Reinforcement learning (RL) agents are powerful tools for managing power grids. They use large amounts of data to inform their actions and receive rewards as feedback to learn favorable responses for the system. This ability is especially valuable in decarbonizing power networks, where the demand for RL agents is increasing.
  arXiv  Detail & Related papers  (2024-11-17T21:30:48Z)
• State and Action Factorization in Power Grids [47.65236082304256]
  We propose a domain-agnostic algorithm that estimates correlations between state and action components entirely based on data. The algorithm is validated on a power grid benchmark obtained with the Grid2Op simulator.
  arXiv  Detail & Related papers  (2024-09-03T15:00:58Z)
• Managing power grids through topology actions: A comparative study between advanced rule-based and reinforcement learning agents [1.8549313085249322]
  Operation of electricity grids has become increasingly complex due to the current upheaval and the increase in renewable energy production. It has been shown that Reinforcement Learning is an efficient and reliable approach with considerable potential for automatic grid operation. In this article, we analyse the submitted agent from Binbinchen and provide novel strategies to improve the agent, both for the RL and the rule-based approach.
  arXiv  Detail & Related papers  (2023-04-03T07:34:43Z)
• Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers [107.3726071306935]
  We propose a new plug-and-play training framework, SMoE-Dropout, to enable scaling transformers to better accuracy in their full capacity without collapse. SMoE-Dropout consists of a randomly and fixed router network to activate experts and gradually increases the activated expert number as training progresses over time. Our experiments demonstrate the superior performance and substantial computation savings of SMoE-Dropout, compared to dense training baselines with equivalent parameter counts.
  arXiv  Detail & Related papers  (2023-03-02T22:12:51Z)
• Reinforcement Learning Based Power Grid Day-Ahead Planning and AI-Assisted Control [0.27998963147546135]
  We introduce a congestion management approach consisting of a redispatching agent and a machine learning-based optimization agent. Compared to a typical redispatching-only agent, it was able to keep a simulated grid in operation longer while at the same time reducing operational cost. The aim of this paper is to bring this promising technology closer to the real world of power grid operation.
  arXiv  Detail & Related papers  (2023-02-15T13:38:40Z)
• Intelligence Processing Units Accelerate Neuromorphic Learning [52.952192990802345]
  Spiking neural networks (SNNs) have achieved orders of magnitude improvement in terms of energy consumption and latency. We present an IPU-optimized release of our custom SNN Python package, snnTorch.
  arXiv  Detail & Related papers  (2022-11-19T15:44:08Z)
• Training Spiking Neural Networks with Local Tandem Learning [96.32026780517097]
  Spiking neural networks (SNNs) are shown to be more biologically plausible and energy efficient than their predecessors. In this paper, we put forward a generalized learning rule, termed Local Tandem Learning (LTL) We demonstrate rapid network convergence within five training epochs on the CIFAR-10 dataset while having low computational complexity.
  arXiv  Detail & Related papers  (2022-10-10T10:05:00Z)
• Learning Dynamics and Generalization in Reinforcement Learning [59.530058000689884]
  We show theoretically that temporal difference learning encourages agents to fit non-smooth components of the value function early in training. We show that neural networks trained using temporal difference algorithms on dense reward tasks exhibit weaker generalization between states than randomly networks and gradient networks trained with policy methods.
  arXiv  Detail & Related papers  (2022-06-05T08:49:16Z)
• Retrieval-Augmented Reinforcement Learning [63.32076191982944]
  We train a network to map a dataset of past experiences to optimal behavior. The retrieval process is trained to retrieve information from the dataset that may be useful in the current context. We show that retrieval-augmented R2D2 learns significantly faster than the baseline R2D2 agent and achieves higher scores.
  arXiv  Detail & Related papers  (2022-02-17T02:44:05Z)
• Adversarial Training for a Continuous Robustness Control Problem in Power Systems [1.0742675209112622]
  We propose a new adversarial training approach for injecting robustness when designing controllers for upcoming cyber-physical power systems. We model an adversarial framework, propose the implementation of a fixed opponent policy and test it on a L2RPN (Learning to Run a Power Network) environment. Using adversarial testing, we analyze the results of submitted trained agents from the robustness track of the L2RPN competition.
  arXiv  Detail & Related papers  (2020-12-21T14:42:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.