Contingency-constrained economic dispatch with safe reinforcement learning

• URL: http://arxiv.org/abs/2205.06212v3
• Date: Tue, 16 Jul 2024 10:00:04 GMT
• Title: Contingency-constrained economic dispatch with safe reinforcement learning
• Authors: Michael Eichelbeck, Hannah Markgraf, Matthias Althoff,
• Abstract summary: Reinforcement-learning based (RL) controllers can address this challenge, but cannot themselves provide safety guarantees. We propose a formally validated RL controller for economic dispatch. We extend conventional constraints by a time-dependent constraint encoding the islanding contingency. Unsafe actions are projected into the safe action space while leveraging constrained zonotope set representations for computational efficiency.
• Score: 7.133681867718039
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Future power systems will rely heavily on micro grids with a high share of decentralised renewable energy sources and energy storage systems. The high complexity and uncertainty in this context might make conventional power dispatch strategies infeasible. Reinforcement-learning based (RL) controllers can address this challenge, however, cannot themselves provide safety guarantees, preventing their deployment in practice. To overcome this limitation, we propose a formally validated RL controller for economic dispatch. We extend conventional constraints by a time-dependent constraint encoding the islanding contingency. The contingency constraint is computed using set-based backwards reachability analysis and actions of the RL agent are verified through a safety layer. Unsafe actions are projected into the safe action space while leveraging constrained zonotope set representations for computational efficiency. The developed approach is demonstrated on a residential use case using real-world measurements.

Related papers

• Sampling-based Safe Reinforcement Learning for Nonlinear Dynamical Systems [15.863561935347692]
  We develop provably safe and convergent reinforcement learning algorithms for control of nonlinear dynamical systems. Recent advances at the intersection of control and RL follow a two-stage, safety filter approach to enforcing hard safety constraints. We develop a single-stage, sampling-based approach to hard constraint satisfaction that learns RL controllers enjoying classical convergence guarantees.
  arXiv  Detail & Related papers  (2024-03-06T19:39:20Z)
• Handling Long and Richly Constrained Tasks through Constrained Hierarchical Reinforcement Learning [20.280636126917614]
  Safety in goal directed Reinforcement Learning (RL) settings has typically been handled through constraints over trajectories. We propose a (safety) Constrained Search with Hierarchical Reinforcement Learning (CoSHRL) mechanism that combines an upper level constrained search agent with a low-level goal conditioned RL agent. A major advantage of CoSHRL is that it can handle constraints on the cost value distribution and can adjust to flexible constraint thresholds without retraining.
  arXiv  Detail & Related papers  (2023-02-21T12:57:12Z)
• Enforcing Hard Constraints with Soft Barriers: Safe Reinforcement Learning in Unknown Stochastic Environments [84.3830478851369]
  We propose a safe reinforcement learning approach that can jointly learn the environment and optimize the control policy. Our approach can effectively enforce hard safety constraints and significantly outperform CMDP-based baseline methods in system safe rate measured via simulations.
  arXiv  Detail & Related papers  (2022-09-29T20:49:25Z)
• Recursively Feasible Probabilistic Safe Online Learning with Control Barrier Functions [60.26921219698514]
  We introduce a model-uncertainty-aware reformulation of CBF-based safety-critical controllers. We then present the pointwise feasibility conditions of the resulting safety controller. We use these conditions to devise an event-triggered online data collection strategy.
  arXiv  Detail & Related papers  (2022-08-23T05:02:09Z)
• Safe Reinforcement Learning via Confidence-Based Filters [78.39359694273575]
  We develop a control-theoretic approach for certifying state safety constraints for nominal policies learned via standard reinforcement learning techniques. We provide formal safety guarantees, and empirically demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2022-07-04T11:43:23Z)
• Reachability Constrained Reinforcement Learning [6.5158195776494]
  This paper proposes a reachability CRL (RCRL) method by using reachability analysis to characterize the largest feasible sets. We also use the multi-time scale approximation theory to prove that the proposed algorithm converges to a local optimum. Empirical results on different benchmarks such as safe-control-gym and Safety-Gym validate the learned feasible set, the performance in optimal criteria, and constraint satisfaction of RCRL.
  arXiv  Detail & Related papers  (2022-05-16T09:32:45Z)
• Safe Reinforcement Learning for Grid Voltage Control [0.0]
  Under voltage load shedding has been considered as a standard approach to recover the voltage stability of the electric power grid under emergency conditions. In this paper, we discuss a couple of novel safe RL approaches, namely constrained optimization approach and Barrier function-based approach.
  arXiv  Detail & Related papers  (2021-12-02T18:34:50Z)
• Improving Robustness of Reinforcement Learning for Power System Control with Adversarial Training [71.7750435554693]
  We show that several state-of-the-art RL agents proposed for power system control are vulnerable to adversarial attacks. Specifically, we use an adversary Markov Decision Process to learn an attack policy, and demonstrate the potency of our attack. We propose to use adversarial training to increase the robustness of RL agent against attacks and avoid infeasible operational decisions.
  arXiv  Detail & Related papers  (2021-10-18T00:50:34Z)
• Enforcing Policy Feasibility Constraints through Differentiable Projection for Energy Optimization [57.88118988775461]
  We propose PROjected Feasibility (PROF) to enforce convex operational constraints within neural policies. We demonstrate PROF on two applications: energy-efficient building operation and inverter control.
  arXiv  Detail & Related papers  (2021-05-19T01:58:10Z)
• Lyapunov Barrier Policy Optimization [15.364174084072872]
  We propose a new method, LBPO, that uses a Lyapunov-based barrier function to restrict the policy update to a safe set for each training iteration. Our method also allows the user to control the conservativeness of the agent with respect to the constraints in the environment.
  arXiv  Detail & Related papers  (2021-03-16T17:58:27Z)
• Closing the Closed-Loop Distribution Shift in Safe Imitation Learning [80.05727171757454]
  We treat safe optimization-based control strategies as experts in an imitation learning problem. We train a learned policy that can be cheaply evaluated at run-time and that provably satisfies the same safety guarantees as the expert.
  arXiv  Detail & Related papers  (2021-02-18T05:11:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.