A Deep Reinforcement Learning Framework for Continuous Intraday Market Bidding

• URL: http://arxiv.org/abs/2004.05940v1
• Date: Mon, 13 Apr 2020 13:50:13 GMT
• Title: A Deep Reinforcement Learning Framework for Continuous Intraday Market Bidding
• Authors: Ioannis Boukas, Damien Ernst, Thibaut Th\'eate, Adrien Bolland, Alexandre Huynen, Martin Buchwald, Christelle Wynants, Bertrand Corn\'elusse
• Abstract summary: A key component for the successful renewable energy sources integration is the usage of energy storage. We propose a novel modelling framework for the strategic participation of energy storage in the European continuous intraday market. An distributed version of the fitted Q algorithm is chosen for solving this problem due to its sample efficiency. Results indicate that the agent converges to a policy that achieves in average higher total revenues than the benchmark strategy.
• Score: 69.37299910149981
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The large integration of variable energy resources is expected to shift a large part of the energy exchanges closer to real-time, where more accurate forecasts are available. In this context, the short-term electricity markets and in particular the intraday market are considered a suitable trading floor for these exchanges to occur. A key component for the successful renewable energy sources integration is the usage of energy storage. In this paper, we propose a novel modelling framework for the strategic participation of energy storage in the European continuous intraday market where exchanges occur through a centralized order book. The goal of the storage device operator is the maximization of the profits received over the entire trading horizon, while taking into account the operational constraints of the unit. The sequential decision-making problem of trading in the intraday market is modelled as a Markov Decision Process. An asynchronous distributed version of the fitted Q iteration algorithm is chosen for solving this problem due to its sample efficiency. The large and variable number of the existing orders in the order book motivates the use of high-level actions and an alternative state representation. Historical data are used for the generation of a large number of artificial trajectories in order to address exploration issues during the learning process. The resulting policy is back-tested and compared against a benchmark strategy that is the current industrial standard. Results indicate that the agent converges to a policy that achieves in average higher total revenues than the benchmark strategy.

Related papers

• Commodification of Compute [0.0]
  This paper introduces a novel global platform for the commodification of compute hours, termed the Global Compute Exchange (GCX) The GCX creates a secure, transparent, and efficient marketplace for buying and selling computational power. This platform aims to revolutionize the computational resource market by fostering a decentralized, efficient, and transparent ecosystem.
  arXiv  Detail & Related papers  (2024-06-27T15:32:31Z)
• Optimizing Quantile-based Trading Strategies in Electricity Arbitrage [0.0]
  This study delves into the optimization of day-ahead and balancing market trading, leveraging quantile-based forecasts. Our findings underscore the profit potential of simultaneous participation in both day-ahead and balancing markets. Despite increased costs and narrower profit margins associated with higher-volume trading, the implementation of high-frequency strategies plays a significant role in maximizing profits.
  arXiv  Detail & Related papers  (2024-06-19T21:27:12Z)
• Stockformer: A Price-Volume Factor Stock Selection Model Based on Wavelet Transform and Multi-Task Self-Attention Networks [3.7608255115473592]
  This paper introduces Stockformer, a price-volume factor stock selection model that integrates wavelet transformation and a multitask self-attention network. Stockformer decomposes stock returns into high and low frequencies, meticulously capturing long-term market trends and abrupt events. Experimental results show that Stockformer outperforms existing advanced methods on multiple real stock market datasets.
  arXiv  Detail & Related papers  (2023-11-23T04:33:47Z)
• HireVAE: An Online and Adaptive Factor Model Based on Hierarchical and Regime-Switch VAE [113.47287249524008]
  It is still an open question to build a factor model that can conduct stock prediction in an online and adaptive setting. We propose the first deep learning based online and adaptive factor model, HireVAE, at the core of which is a hierarchical latent space that embeds the relationship between the market situation and stock-wise latent factors. Across four commonly used real stock market benchmarks, the proposed HireVAE demonstrate superior performance in terms of active returns over previous methods.
  arXiv  Detail & Related papers  (2023-06-05T12:58:13Z)
• Market Making with Scaled Beta Policies [2.064612766965483]
  This paper introduces a new representation for the actions of a market maker in an order-driven market. It uses scaled beta distributions, and generalises three approaches taken in the artificial intelligence for market making literature. We demonstrate that this flexibility is useful for inventory management, one of the key challenges faced by a market maker.
  arXiv  Detail & Related papers  (2022-07-07T14:59:34Z)
• Concepts and Algorithms for Agent-based Decentralized and Integrated Scheduling of Production and Auxiliary Processes [78.120734120667]
  This paper describes an agent-based decentralized and integrated scheduling approach. Part of the requirements is to develop a linearly scaling communication architecture. The approach is explained using an example based on industrial requirements.
  arXiv  Detail & Related papers  (2022-05-06T18:44:29Z)
• Deep Q-Learning Market Makers in a Multi-Agent Simulated Stock Market [58.720142291102135]
  This paper focuses precisely on the study of these markets makers strategies from an agent-based perspective. We propose the application of Reinforcement Learning (RL) for the creation of intelligent market markers in simulated stock markets.
  arXiv  Detail & Related papers  (2021-12-08T14:55:21Z)
• A Reinforcement Learning Approach for the Continuous Electricity Market of Germany: Trading from the Perspective of a Wind Park Operator [0.0]
  We propose a novel autonomous trading approach based on Deep Reinforcement Learning (DRL) algorithms. We test our framework in a case study from the perspective of a wind park operator.
  arXiv  Detail & Related papers  (2021-11-26T17:17:27Z)
• ARISE: ApeRIodic SEmi-parametric Process for Efficient Markets without Periodogram and Gaussianity Assumptions [91.3755431537592]
  We present the ApeRI-miodic (ARISE) process for investigating efficient markets. The ARISE process is formulated as an infinite-sum of some known processes and employs the aperiodic spectrum estimation. In practice, we apply the ARISE function to identify the efficiency of real-world markets.
  arXiv  Detail & Related papers  (2021-11-08T03:36:06Z)
• Universal Trading for Order Execution with Oracle Policy Distillation [99.57416828489568]
  We propose a novel universal trading policy optimization framework to bridge the gap between the noisy yet imperfect market states and the optimal action sequences for order execution. We show that our framework can better guide the learning of the common policy towards practically optimal execution by an oracle teacher with perfect information.
  arXiv  Detail & Related papers  (2021-01-28T05:52:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.