On-line reinforcement learning for optimization of real-life energy trading strategy

• URL: http://arxiv.org/abs/2303.16266v3
• Date: Wed, 14 Feb 2024 15:34:10 GMT
• Title: On-line reinforcement learning for optimization of real-life energy trading strategy
• Authors: {\L}ukasz Lepak and Pawe{\l} Wawrzy\'nski
• Abstract summary: This paper considers automated trading on the day-ahead (DA) energy market by a medium-sized prosumer. We formalize a framework in which an applicable in real-life strategy can be optimized with off-line data. We use state-of-the-art reinforcement learning (RL) algorithms to optimize this strategy.
• Score: 0.9790236766474201
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: An increasing share of energy is produced from renewable sources by many small producers. The efficiency of those sources is volatile and, to some extent, random, exacerbating the problem of energy market balancing. In many countries, this balancing is done on the day-ahead (DA) energy markets. This paper considers automated trading on the DA energy market by a medium-sized prosumer. We model this activity as a Markov Decision Process and formalize a framework in which an applicable in real-life strategy can be optimized with off-line data. We design a trading strategy that is fed with the available environmental information that can impact future prices, including weather forecasts. We use state-of-the-art reinforcement learning (RL) algorithms to optimize this strategy. For comparison, we also synthesize simple parametric trading strategies and optimize them with an evolutionary algorithm. Results show that our RL-based strategy generates the highest market profits.

Related papers

• EPO: Explicit Policy Optimization for Strategic Reasoning in LLMs via Reinforcement Learning [69.55982246413046]
  We propose explicit policy optimization (EPO) for strategic reasoning. EPO provides strategies in open-ended action space and can be plugged into arbitrary LLM agents to motivate goal-directed behavior. Experiments across social and physical domains demonstrate EPO's ability of long-term goal alignment.
  arXiv  Detail & Related papers  (2025-02-18T03:15:55Z)
• Deep Reinforcement Learning Agents for Strategic Production Policies in Microeconomic Market Simulations [1.6499388997661122]
  We propose a DRL-based approach to obtain an effective policy in competitive markets with multiple producers. Our framework enables agents to learn adaptive production policies to several simulations that consistently outperform static and random strategies. The results show that agents trained with DRL can strategically adjust production levels to maximize long-term profitability.
  arXiv  Detail & Related papers  (2024-10-27T18:38:05Z)
• Deep Reinforcement Learning for Online Optimal Execution Strategies [49.1574468325115]
  This paper tackles the challenge of learning non-Markovian optimal execution strategies in dynamic financial markets. We introduce a novel actor-critic algorithm based on Deep Deterministic Policy Gradient (DDPG) We show that our algorithm successfully approximates the optimal execution strategy.
  arXiv  Detail & Related papers  (2024-10-17T12:38:08Z)
• Interpretable Deep Reinforcement Learning for Optimizing Heterogeneous Energy Storage Systems [11.03157076666012]
  Energy storage systems (ESS) are pivotal component in the energy market, serving as both energy suppliers and consumers. To enhance ESS flexibility within the energy market, a heterogeneous photovoltaic-ESS (PV-ESS) is proposed. We develop a comprehensive cost function that takes into account degradation, capital, and operation/maintenance costs to reflect real-world scenarios.
  arXiv  Detail & Related papers  (2023-10-20T02:26:17Z)
• Hybrid Reinforcement Learning for Optimizing Pump Sustainability in Real-World Water Distribution Networks [55.591662978280894]
  This article addresses the pump-scheduling optimization problem to enhance real-time control of real-world water distribution networks (WDNs) Our primary objectives are to adhere to physical operational constraints while reducing energy consumption and operational costs. Traditional optimization techniques, such as evolution-based and genetic algorithms, often fall short due to their lack of convergence guarantees.
  arXiv  Detail & Related papers  (2023-10-13T21:26:16Z)
• Multi-market Energy Optimization with Renewables via Reinforcement Learning [1.0878040851638]
  This paper introduces a deep reinforcement learning framework for optimizing the operations of power plants pairing renewable energy with storage. The framework handles complexities such as time coupling by storage devices, uncertainty in renewable generation and energy prices, and non-linear storage models. It utilizes RL to incorporate complex storage models, overcoming restrictions of optimization-based methods that require convex and differentiable component models.
  arXiv  Detail & Related papers  (2023-06-13T21:35:24Z)
• Optimal Planning of Hybrid Energy Storage Systems using Curtailed Renewable Energy through Deep Reinforcement Learning [0.0]
  We propose a sophisticated deep reinforcement learning (DRL) methodology with a policy-based algorithm to plan energy storage systems (ESS) A quantitative performance comparison proved that the DRL agent outperforms the scenario-based optimization (SO) algorithm. The corresponding results confirmed that the DRL agent learns the way like what a human expert would do, suggesting reliable application of the proposed methodology.
  arXiv  Detail & Related papers  (2022-12-12T02:24:50Z)
• Sequential Information Design: Markov Persuasion Process and Its Efficient Reinforcement Learning [156.5667417159582]
  This paper proposes a novel model of sequential information design, namely the Markov persuasion processes (MPPs) Planning in MPPs faces the unique challenge in finding a signaling policy that is simultaneously persuasive to the myopic receivers and inducing the optimal long-term cumulative utilities of the sender. We design a provably efficient no-regret learning algorithm, the Optimism-Pessimism Principle for Persuasion Process (OP4), which features a novel combination of both optimism and pessimism principles.
  arXiv  Detail & Related papers  (2022-02-22T05:41:43Z)
• Deep Q-Learning Market Makers in a Multi-Agent Simulated Stock Market [58.720142291102135]
  This paper focuses precisely on the study of these markets makers strategies from an agent-based perspective. We propose the application of Reinforcement Learning (RL) for the creation of intelligent market markers in simulated stock markets.
  arXiv  Detail & Related papers  (2021-12-08T14:55:21Z)
• A Reinforcement Learning Approach for the Continuous Electricity Market of Germany: Trading from the Perspective of a Wind Park Operator [0.0]
  We propose a novel autonomous trading approach based on Deep Reinforcement Learning (DRL) algorithms. We test our framework in a case study from the perspective of a wind park operator.
  arXiv  Detail & Related papers  (2021-11-26T17:17:27Z)
• Universal Trading for Order Execution with Oracle Policy Distillation [99.57416828489568]
  We propose a novel universal trading policy optimization framework to bridge the gap between the noisy yet imperfect market states and the optimal action sequences for order execution. We show that our framework can better guide the learning of the common policy towards practically optimal execution by an oracle teacher with perfect information.
  arXiv  Detail & Related papers  (2021-01-28T05:52:18Z)
• A Deep Reinforcement Learning Framework for Continuous Intraday Market Bidding [69.37299910149981]
  A key component for the successful renewable energy sources integration is the usage of energy storage. We propose a novel modelling framework for the strategic participation of energy storage in the European continuous intraday market. An distributed version of the fitted Q algorithm is chosen for solving this problem due to its sample efficiency. Results indicate that the agent converges to a policy that achieves in average higher total revenues than the benchmark strategy.
  arXiv  Detail & Related papers  (2020-04-13T13:50:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.