Comparing Traditional and Reinforcement-Learning Methods for Energy Storage Control

• URL: http://arxiv.org/abs/2506.00459v1
• Date: Sat, 31 May 2025 08:25:21 GMT
• Title: Comparing Traditional and Reinforcement-Learning Methods for Energy Storage Control
• Authors: Elinor Ginzburg, Itay Segev, Yoash Levron, Sarah Keren,
• Abstract summary: We aim to better understand the tradeoffs between traditional and reinforcement learning approaches for energy storage management.<n>We compare the performance of traditional and RL methods, discuss settings in which it is beneficial to use each method, and suggest avenues for future investigation.
• Score: 6.452975320319021
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We aim to better understand the tradeoffs between traditional and reinforcement learning (RL) approaches for energy storage management. More specifically, we wish to better understand the performance loss incurred when using a generative RL policy instead of using a traditional approach to find optimal control policies for specific instances. Our comparison is based on a simplified micro-grid model, that includes a load component, a photovoltaic source, and a storage device. Based on this model, we examine three use cases of increasing complexity: ideal storage with convex cost functions, lossy storage devices, and lossy storage devices with convex transmission losses. With the aim of promoting the principled use RL based methods in this challenging and important domain, we provide a detailed formulation of each use case and a detailed description of the optimization challenges. We then compare the performance of traditional and RL methods, discuss settings in which it is beneficial to use each method, and suggest avenues for future investigation.

Related papers

• A Reinforcement Learning Approach for Optimal Control in Microgrids [43.122212629962235]
  Microgrids provide a promising solution by enabling localized control over energy generation, storage, and distribution.<n>This paper presents a novel reinforcement learning (RL)-based methodology for optimizing microgrid energy management.
  arXiv  Detail & Related papers  (2025-06-28T20:10:00Z)
• Accelerating RL for LLM Reasoning with Optimal Advantage Regression [52.0792918455501]
  We propose a novel two-stage policy optimization framework that directly approximates the optimal advantage function.<n>$A$*-PO achieves competitive performance across a wide range of mathematical reasoning benchmarks.<n>It reduces training time by up to 2$times$ and peak memory usage by over 30% compared to PPO, GRPO, and REBEL.
  arXiv  Detail & Related papers  (2025-05-27T03:58:50Z)
• Preference Optimization for Combinatorial Optimization Problems [54.87466279363487]
  Reinforcement Learning (RL) has emerged as a powerful tool for neural optimization, enabling models learns that solve complex problems without requiring expert knowledge.<n>Despite significant progress, existing RL approaches face challenges such as diminishing reward signals and inefficient exploration in vast action spaces.<n>We propose Preference Optimization, a novel method that transforms quantitative reward signals into qualitative preference signals via statistical comparison modeling.
  arXiv  Detail & Related papers  (2025-05-13T16:47:00Z)
• PearSAN: A Machine Learning Method for Inverse Design using Pearson Correlated Surrogate Annealing [66.27103948750306]
  PearSAN is a machine learning-assisted optimization algorithm applicable to inverse design problems with large design spaces.<n>It uses a Pearson correlated surrogate model to predict the figure of merit of the true design metric.<n>It achieves a state-of-the-art maximum design efficiency of 97%, and is at least an order of magnitude faster than previous methods.
  arXiv  Detail & Related papers  (2024-12-26T17:02:19Z)
• Distill2Explain: Differentiable decision trees for explainable reinforcement learning in energy application controllers [5.311053322050159]
  Residential sector is an important (potential) source of energy flexibility. A potential control framework for such a task is data-driven control, specifically model-free reinforcement learning (RL) RLs learn a good control policy by interacting with their environment, learning purely based on data and with minimal human intervention. We propose a novel method to obtain explainable RL policies by using differentiable decision trees.
  arXiv  Detail & Related papers  (2024-03-18T16:09:49Z)
• Interpretable Deep Reinforcement Learning for Optimizing Heterogeneous Energy Storage Systems [11.03157076666012]
  Energy storage systems (ESS) are pivotal component in the energy market, serving as both energy suppliers and consumers. To enhance ESS flexibility within the energy market, a heterogeneous photovoltaic-ESS (PV-ESS) is proposed. We develop a comprehensive cost function that takes into account degradation, capital, and operation/maintenance costs to reflect real-world scenarios.
  arXiv  Detail & Related papers  (2023-10-20T02:26:17Z)
• Hybrid Reinforcement Learning for Optimizing Pump Sustainability in Real-World Water Distribution Networks [55.591662978280894]
  This article addresses the pump-scheduling optimization problem to enhance real-time control of real-world water distribution networks (WDNs) Our primary objectives are to adhere to physical operational constraints while reducing energy consumption and operational costs. Traditional optimization techniques, such as evolution-based and genetic algorithms, often fall short due to their lack of convergence guarantees.
  arXiv  Detail & Related papers  (2023-10-13T21:26:16Z)
• Multi-market Energy Optimization with Renewables via Reinforcement Learning [1.0878040851638]
  This paper introduces a deep reinforcement learning framework for optimizing the operations of power plants pairing renewable energy with storage. The framework handles complexities such as time coupling by storage devices, uncertainty in renewable generation and energy prices, and non-linear storage models. It utilizes RL to incorporate complex storage models, overcoming restrictions of optimization-based methods that require convex and differentiable component models.
  arXiv  Detail & Related papers  (2023-06-13T21:35:24Z)
• Mastering the Unsupervised Reinforcement Learning Benchmark from Pixels [112.63440666617494]
  Reinforcement learning algorithms can succeed but require large amounts of interactions between the agent and the environment. We propose a new method to solve it, using unsupervised model-based RL, for pre-training the agent. We show robust performance on the Real-Word RL benchmark, hinting at resiliency to environment perturbations during adaptation.
  arXiv  Detail & Related papers  (2022-09-24T14:22:29Z)
• Multi-fidelity reinforcement learning framework for shape optimization [0.8258451067861933]
  We introduce a controlled transfer learning framework that leverages a multi-fidelity simulation setting. Our strategy is deployed for an airfoil shape optimization problem at high Reynolds numbers. Our results demonstrate this framework's applicability to other scientific DRL scenarios.
  arXiv  Detail & Related papers  (2022-02-22T20:44:04Z)
• An Experimental Design Perspective on Model-Based Reinforcement Learning [73.37942845983417]
  In practical applications of RL, it is expensive to observe state transitions from the environment. We propose an acquisition function that quantifies how much information a state-action pair would provide about the optimal solution to a Markov decision process.
  arXiv  Detail & Related papers  (2021-12-09T23:13:57Z)
• Robust Predictable Control [149.71263296079388]
  We show that our method achieves much tighter compression than prior methods, achieving up to 5x higher reward than a standard information bottleneck. We also demonstrate that our method learns policies that are more robust and generalize better to new tasks.
  arXiv  Detail & Related papers  (2021-09-07T17:29:34Z)
• Deep Controlled Learning for Inventory Control [0.0]
  Controlled Deep Learning (DCL) is a new DRL framework based on approximate policy specifically designed to tackle inventory problems. DCL outperforms existing state-of-the-art iterations in lost sales inventory control, perishable inventory systems, and inventory systems with random lead times. These substantial performance and robustness improvements pave the way for the effective application of tailored DRL algorithms to inventory management problems.
  arXiv  Detail & Related papers  (2020-11-30T18:53:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.