How to Use Reinforcement Learning to Facilitate Future Electricity Market Design? Part 2: Method and Applications

• URL: http://arxiv.org/abs/2305.06921v2
• Date: Fri, 12 May 2023 00:47:46 GMT
• Title: How to Use Reinforcement Learning to Facilitate Future Electricity Market Design? Part 2: Method and Applications
• Authors: Ziqing Zhu, Siqi Bu, Ka Wing Chan, Bin Zhou, Shiwei Xia
• Abstract summary: This paper develops a paradigmatic theory and detailed methods of the joint electricity market design using reinforcement-learning (RL)-based simulation. The Markov game model is developed, in which we show how to incorporate market design options and uncertain risks in model formulation. A multi-agent policy proximal optimization (MAPPO) algorithm is elaborated, as a practical implementation of the generalized market simulation method developed in Part 1.
• Score: 7.104195252081324
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This two-part paper develops a paradigmatic theory and detailed methods of the joint electricity market design using reinforcement-learning (RL)-based simulation. In Part 2, this theory is further demonstrated by elaborating detailed methods of designing an electricity spot market (ESM), together with a reserved capacity product (RC) in the ancillary service market (ASM) and a virtual bidding (VB) product in the financial market (FM). Following the theory proposed in Part 1, firstly, market design options in the joint market are specified. Then, the Markov game model is developed, in which we show how to incorporate market design options and uncertain risks in model formulation. A multi-agent policy proximal optimization (MAPPO) algorithm is elaborated, as a practical implementation of the generalized market simulation method developed in Part 1. Finally, the case study demonstrates how to pick the best market design options by using some of the market operation performance indicators proposed in Part 1, based on the simulation results generated by implementing the MAPPO algorithm. The impacts of different market design options on market participants' bidding strategy preference are also discussed.

Related papers

• There is No Silver Bullet: Benchmarking Methods in Predictive Combinatorial Optimization [59.27851754647913]
  Predictive optimization is the precise modeling of many real-world applications, including energy cost-aware scheduling and budget allocation on advertising. There is no systematic benchmark of both approaches, including the specific design choices at the module level. Our study shows that PnO approaches are better than PtO on 7 out of 8 benchmarks, but there is no silver bullet found for the specific design choices of PnO.
  arXiv  Detail & Related papers  (2023-11-13T13:19:34Z)
• HireVAE: An Online and Adaptive Factor Model Based on Hierarchical and Regime-Switch VAE [113.47287249524008]
  It is still an open question to build a factor model that can conduct stock prediction in an online and adaptive setting. We propose the first deep learning based online and adaptive factor model, HireVAE, at the core of which is a hierarchical latent space that embeds the relationship between the market situation and stock-wise latent factors. Across four commonly used real stock market benchmarks, the proposed HireVAE demonstrate superior performance in terms of active returns over previous methods.
  arXiv  Detail & Related papers  (2023-06-05T12:58:13Z)
• How to Use Reinforcement Learning to Facilitate Future Electricity Market Design? Part 1: A Paradigmatic Theory [7.104195252081324]
  This paper develops a paradigmatic theory and detailed methods of the joint market design using reinforcement-learning (RL)-based simulation. Several market operation performance indicators are proposed to validate the market design based on the simulation results.
  arXiv  Detail & Related papers  (2023-05-04T01:30:15Z)
• Approximating Energy Market Clearing and Bidding With Model-Based Reinforcement Learning [0.0]
  Multi-agent Reinforcement learning (MARL) is a promising new approach to predicting the expected profit-maximizing behavior of energy market participants in simulation. We provide a model of the energy market to a basic MARL algorithm in the form of a learned OPF approximation and explicit market rules. Our experiments demonstrate that the model reduces training time by about one order of magnitude but at the cost of a slightly worse performance.
  arXiv  Detail & Related papers  (2023-03-03T08:26:22Z)
• Market-Aware Models for Efficient Cross-Market Recommendation [14.663751660438729]
  Cross-market recommendation (CMR) involves recommendation in a low-resource target market using data from a richer, auxiliary source market. Prior work in CMR utilised meta-learning to improve recommendation performance in target markets. We propose market-aware (MA) models, which directly model a market via market embeddings instead of meta-learning.
  arXiv  Detail & Related papers  (2023-02-14T15:44:22Z)
• Deep Q-Learning Market Makers in a Multi-Agent Simulated Stock Market [58.720142291102135]
  This paper focuses precisely on the study of these markets makers strategies from an agent-based perspective. We propose the application of Reinforcement Learning (RL) for the creation of intelligent market markers in simulated stock markets.
  arXiv  Detail & Related papers  (2021-12-08T14:55:21Z)
• Towards Realistic Market Simulations: a Generative Adversarial Networks Approach [2.381990157809543]
  We propose a synthetic market generator based on Conditional Generative Adversarial Networks (CGANs) trained on real aggregate-level historical data. A CGAN-based "world" agent can generate meaningful orders in response to an experimental agent.
  arXiv  Detail & Related papers  (2021-10-25T22:01:07Z)
• Estimating Fund-Raising Performance for Start-up Projects from a Market Graph Perspective [58.353799280109904]
  We propose a Graph-based Market Environment (GME) model for predicting the fund-raising performance of the unpublished project by exploiting the market environment. Specifically, we propose a Graph-based Market Environment (GME) model for predicting the fund-raising performance of the unpublished project by exploiting the market environment.
  arXiv  Detail & Related papers  (2021-05-27T02:39:30Z)
• Markdowns in E-Commerce Fresh Retail: A Counterfactual Prediction and Multi-Period Optimization Approach [29.11201102550876]
  We build a semi-parametric structural model to learn individual price elasticity and predict counterfactual demand. We propose a multi-period dynamic pricing algorithm to maximize the overall profit of a perishable product over its finite selling horizon. The proposed framework has been successfully deployed to the well-known e-commerce fresh retail scenario - Freshippo.
  arXiv  Detail & Related papers  (2021-05-18T07:01:37Z)
• Universal Trading for Order Execution with Oracle Policy Distillation [99.57416828489568]
  We propose a novel universal trading policy optimization framework to bridge the gap between the noisy yet imperfect market states and the optimal action sequences for order execution. We show that our framework can better guide the learning of the common policy towards practically optimal execution by an oracle teacher with perfect information.
  arXiv  Detail & Related papers  (2021-01-28T05:52:18Z)
• Energy-based View of Retrosynthesis [70.66156081030766]
  We propose a framework that unifies sequence- and graph-based methods as energy-based models. We present a novel dual variant within the framework that performs consistent training over Bayesian forward- and backward-prediction. This model improves state-of-the-art performance by 9.6% for template-free approaches where the reaction type is unknown.
  arXiv  Detail & Related papers  (2020-07-14T18:51:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.