Physics Informed Reinforcement Learning with Gibbs Priors for Topology Control in Power Grids

• URL: http://arxiv.org/abs/2604.01830v1
• Date: Thu, 02 Apr 2026 09:45:35 GMT
• Title: Physics Informed Reinforcement Learning with Gibbs Priors for Topology Control in Power Grids
• Abstract summary: Topology control for power grid operation is a challenging sequential decision making problem.<n>We propose a physics-informed Reinforcement Learning framework that encodes the system's physics, over the action space.<n>We evaluate the approach in three realistic benchmark environments of increasing difficulty.
• Score: 11.77206222026137
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Topology control for power grid operation is a challenging sequential decision making problem because the action space grows combinatorially with the size of the grid and action evaluation through simulation is computationally expensive. We propose a physics-informed Reinforcement Learning framework that combines semi-Markov control with a Gibbs prior, that encodes the system's physics, over the action space. The decision is only taken when the grid enters a hazardous regime, while a graph neural network surrogate predicts the post action overload risk of feasible topology actions. These predictions are used to construct a physics-informed Gibbs prior that both selects a small state-dependent candidate set and reweights policy logits before action selection. In this way, our method reduces exploration difficulty and online simulation cost while preserving the flexibility of a learned policy. We evaluate the approach in three realistic benchmark environments of increasing difficulty. Across all settings, the proposed method achieves a strong balance between control quality and computational efficiency: it matches oracle-level performance while being approximately $6\times$ faster on the first benchmark, reaches $94.6\%$ of oracle reward with roughly $200\times$ lower decision time on the second one, and on the most challenging benchmark improves over a PPO baseline by up to $255\%$ in reward and $284\%$ in survived steps while remaining about $2.5\times$ faster than a strong specialized engineering baseline. These results show that our method provides an effective mechanism for topology control in power grids.

Related papers

• Learning Not to Optimize: Physics-Informed Action-Space Reshaping for Intent-Based Network Control [50.46501177871741]
  We show that LNOQRD reduces small-instance candidates by $75.9%$ while retaining $90.8%$ near-oracle coverage.<n>Experiments show that LNOQRD achieves the highest utility and intent satisfaction, the lowest hard-law violation and post-generation latency.
  arXiv  Detail & Related papers  (2026-08-02T00:27:01Z)
• Robust Deterministic Policy Gradient for Disturbance Attenuation and Its Application to Quadrotor Control [6.729634947284263]
  We introduce an algorithm called robust deep deterministic policy gradient (RDDPG), which employs a deep neural network architecture and integrates techniques from the twin-delayed deep deterministic policy gradient (TD3) to enhance stability and learning efficiency.<n>The proposed algorithm outperforms other control approaches in terms of robustness against disturbances, enabling precise real-time tracking of moving targets even under severe disturbance conditions.
  arXiv  Detail & Related papers  (2025-02-28T13:58:22Z)
• Predictive Linear Online Tracking for Unknown Targets [5.047136039782827]
  We study the problem of online tracking in linear control systems, where the objective is to follow a moving target. We propose a new algorithm, called predictive linear online tracking (PLOT) We implement PLOT on a real quadrotor and provide open-source software, thus, showcasing one of the first successful applications of online control methods on real hardware.
  arXiv  Detail & Related papers  (2024-02-15T15:59:59Z)
• Drift Control of High-Dimensional RBM: A Computational Method Based on Neural Networks [4.540236408836132]
  We develop and illustrate a simulation-based computational method that relies heavily on deep neural network technology. For test problems studied thus far, our method is accurate to within a fraction of one percent, and is computationally feasible in dimensions up to at least $d=30$.
  arXiv  Detail & Related papers  (2023-09-20T21:32:58Z)
• TransPath: Learning Heuristics For Grid-Based Pathfinding via Transformers [64.88759709443819]
  We suggest learning the instance-dependent proxies that are supposed to notably increase the efficiency of the search. The first proxy we suggest to learn is the correction factor, i.e. the ratio between the instance independent cost-to-go estimate and the perfect one. The second proxy is the path probability, which indicates how likely the grid cell is lying on the shortest path.
  arXiv  Detail & Related papers  (2022-12-22T14:26:11Z)
• Actor-Critic based Improper Reinforcement Learning [61.430513757337486]
  We consider an improper reinforcement learning setting where a learner is given $M$ base controllers for an unknown Markov decision process. We propose two algorithms: (1) a Policy Gradient-based approach; and (2) an algorithm that can switch between a simple Actor-Critic scheme and a Natural Actor-Critic scheme.
  arXiv  Detail & Related papers  (2022-07-19T05:55:02Z)
• Dealing with Sparse Rewards in Continuous Control Robotics via Heavy-Tailed Policies [64.2210390071609]
  We present a novel Heavy-Tailed Policy Gradient (HT-PSG) algorithm to deal with the challenges of sparse rewards in continuous control problems. We show consistent performance improvement across all tasks in terms of high average cumulative reward.
  arXiv  Detail & Related papers  (2022-06-12T04:09:39Z)
• Safe Adaptive Learning-based Control for Constrained Linear Quadratic Regulators with Regret Guarantees [11.627320138064684]
  We study the adaptive control of an unknown linear system with a quadratic cost function subject to safety constraints on both the states and actions. Our algorithm is implemented on a single trajectory and does not require system restarts.
  arXiv  Detail & Related papers  (2021-10-31T05:52:42Z)
• Deep $\mathcal{L}^1$ Stochastic Optimal Control Policies for Planetary Soft-landing [9.714390258486569]
  We introduce a novel deep learning based solution to the Powered-Descent Guidance (PDG) problem. Our SOC can handle practically useful $mathcalL1 constraints pre-specified for minimum fuel consumption. We demonstrate that our controller can successfully and safely land all trajectories at the base of an inverted cone while minimizing fuel consumption.
  arXiv  Detail & Related papers  (2021-09-01T04:28:38Z)
• Non-stationary Online Learning with Memory and Non-stochastic Control [71.14503310914799]
  We study the problem of Online Convex Optimization (OCO) with memory, which allows loss functions to depend on past decisions. In this paper, we introduce dynamic policy regret as the performance measure to design algorithms robust to non-stationary environments. We propose a novel algorithm for OCO with memory that provably enjoys an optimal dynamic policy regret in terms of time horizon, non-stationarity measure, and memory length.
  arXiv  Detail & Related papers  (2021-02-07T09:45:15Z)
• Momentum-Based Policy Gradient Methods [133.53164856723782]
  We propose a class of efficient momentum-based policy gradient methods for the model-free reinforcement learning. In particular, we present a non-adaptive version of IS-MBPG method, which also reaches the best known sample complexity of $O(epsilon-3)$ without any large batches.
  arXiv  Detail & Related papers  (2020-07-13T20:44:15Z)
• Adaptive Control and Regret Minimization in Linear Quadratic Gaussian (LQG) Setting [91.43582419264763]
  We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty. LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model.
  arXiv  Detail & Related papers  (2020-03-12T19:56:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.