High-dimensional Bid Learning for Energy Storage Bidding in Energy Markets

• URL: http://arxiv.org/abs/2311.02551v1
• Date: Sun, 5 Nov 2023 02:59:53 GMT
• Title: High-dimensional Bid Learning for Energy Storage Bidding in Energy Markets
• Authors: Jinyu Liu, Hongye Guo, Qinghu Tang, En Lu, Qiuna Cai, Qixin Chen
• Abstract summary: We propose a new bid representation method called Neural Network Embedded Bids (NNEBs) Our studies show that the proposed method achieves 18% higher profit than the baseline and up to 78% profit of the optimal market bidder.
• Score: 2.1053035142861423
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the growing penetration of renewable energy resource, electricity market prices have exhibited greater volatility. Therefore, it is important for Energy Storage Systems(ESSs) to leverage the multidimensional nature of energy market bids to maximize profitability. However, current learning methods cannot fully utilize the high-dimensional price-quantity bids in the energy markets. To address this challenge, we modify the common reinforcement learning(RL) process by proposing a new bid representation method called Neural Network Embedded Bids (NNEBs). NNEBs refer to market bids that are represented by monotonic neural networks with discrete outputs. To achieve effective learning of NNEBs, we first learn a neural network as a strategic mapping from the market price to ESS power output with RL. Then, we re-train the network with two training modifications to make the network output monotonic and discrete. Finally, the neural network is equivalently converted into a high-dimensional bid for bidding. We conducted experiments over real-world market datasets. Our studies show that the proposed method achieves 18% higher profit than the baseline and up to 78% profit of the optimal market bidder.

Related papers

• Reinforcement Learning Based Bidding Framework with High-dimensional Bids in Power Markets [3.8066343577384796]
  We propose a framework to fully utilize HDBs for RL-based bidding methods. First, we employ a special type of neural network called Neural Network Supply Functions (NNSFs) to generate HDBs in the form of N price-power pairs. Second, we embed the NNSF into a Markov Decision Process (MDP) to make it compatible with most existing RL methods.
  arXiv  Detail & Related papers  (2024-10-15T01:39:28Z)
• Towards Certified Unlearning for Deep Neural Networks [50.816473152067104]
  certified unlearning has been extensively studied in convex machine learning models. We propose several techniques to bridge the gap between certified unlearning and deep neural networks (DNNs)
  arXiv  Detail & Related papers  (2024-08-01T21:22:10Z)
• Temporal-Aware Deep Reinforcement Learning for Energy Storage Bidding in Energy and Contingency Reserve Markets [13.03742132147551]
  We develop a novel BESS joint bidding strategy that utilizes deep reinforcement learning (DRL) to bid in the spot and contingency frequency control ancillary services markets. Unlike conventional "black-box" DRL model, our approach is more interpretable and provides valuable insights into the temporal bidding behavior of BESS.
  arXiv  Detail & Related papers  (2024-02-29T12:41:54Z)
• Exploring the Privacy-Energy Consumption Tradeoff for Split Federated Learning [51.02352381270177]
  Split Federated Learning (SFL) has recently emerged as a promising distributed learning technology. The choice of the cut layer in SFL can have a substantial impact on the energy consumption of clients and their privacy. This article provides a comprehensive overview of the SFL process and thoroughly analyze energy consumption and privacy.
  arXiv  Detail & Related papers  (2023-11-15T23:23:42Z)
• Cryptocurrency Portfolio Optimization by Neural Networks [81.20955733184398]
  This paper proposes an effective algorithm based on neural networks to take advantage of these investment products. A deep neural network, which outputs the allocation weight of each asset at a time interval, is trained to maximize the Sharpe ratio. A novel loss term is proposed to regulate the network's bias towards a specific asset, thus enforcing the network to learn an allocation strategy that is close to a minimum variance strategy.
  arXiv  Detail & Related papers  (2023-10-02T12:33:28Z)
• Transferable Energy Storage Bidder [0.0]
  This paper presents a novel, versatile, and transferable approach combining model-based optimization with a convolutional long short-term memory network for energy storage. We test our proposed approach using historical prices from New York State, showing it achieves state-of-the-art results. We also test a transfer learning approach by pre-training the bidding model using New York data and applying it to arbitrage in Queensland, Australia.
  arXiv  Detail & Related papers  (2023-01-02T01:04:02Z)
• Proximal Policy Optimization Based Reinforcement Learning for Joint Bidding in Energy and Frequency Regulation Markets [6.175137568373435]
  Energy arbitrage can be a significant source of revenue for the battery energy storage system (BESS) It is crucial for the BESS to carefully decide how much capacity to assign to each market to maximize the total profit under uncertain market conditions. This paper formulates the bidding problem of the BESS as a Markov Decision Process, which enables the BESS to participate in both the spot market and the FCAS market to maximize profit.
  arXiv  Detail & Related papers  (2022-12-13T13:07:31Z)
• Graph Neural Networks for Learning Real-Time Prices in Electricity Market [21.402299307739558]
  We propose a new graph neural network (GNN) framework for predicting the electricity market prices from solving OPFs. The proposed GNN-for-OPF framework innovatively exploits the locality property of prices and introduces physics-aware regularization. Numerical tests have validated the learning efficiency and adaptivity improvements of our proposed method over existing approaches.
  arXiv  Detail & Related papers  (2021-06-19T16:34:56Z)
• A Multi-Agent Deep Reinforcement Learning Approach for a Distributed Energy Marketplace in Smart Grids [58.666456917115056]
  This paper presents a Reinforcement Learning based energy market for a prosumer dominated microgrid. The proposed market model facilitates a real-time and demanddependent dynamic pricing environment, which reduces grid costs and improves the economic benefits for prosumers.
  arXiv  Detail & Related papers  (2020-09-23T02:17:51Z)
• Neural Networks and Value at Risk [59.85784504799224]
  We perform Monte-Carlo simulations of asset returns for Value at Risk threshold estimation. Using equity markets and long term bonds as test assets, we investigate neural networks. We find our networks when fed with substantially less data to perform significantly worse.
  arXiv  Detail & Related papers  (2020-05-04T17:41:59Z)
• A Deep Reinforcement Learning Framework for Continuous Intraday Market Bidding [69.37299910149981]
  A key component for the successful renewable energy sources integration is the usage of energy storage. We propose a novel modelling framework for the strategic participation of energy storage in the European continuous intraday market. An distributed version of the fitted Q algorithm is chosen for solving this problem due to its sample efficiency. Results indicate that the agent converges to a policy that achieves in average higher total revenues than the benchmark strategy.
  arXiv  Detail & Related papers  (2020-04-13T13:50:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.