Related papers: Assessing Long-Term Electricity Market Design for Ambitious Decarbonization Targets using Multi-Agent Reinforcement Learning

Assessing Long-Term Electricity Market Design for Ambitious Decarbonization Targets using Multi-Agent Reinforcement Learning

URL: http://arxiv.org/abs/2512.17444v1
Date: Fri, 19 Dec 2025 10:56:34 GMT
Title: Assessing Long-Term Electricity Market Design for Ambitious Decarbonization Targets using Multi-Agent Reinforcement Learning
Authors: Javier Gonzalez-Ruiz, Carlos Rodriguez-Pardo, Iacopo Savelli, Alice Di Bella, Massimo Tavoni,
Abstract summary: This work presents a multi-agent reinforcement learning model capable of capturing the key features of decarbonizing energy systems.<n>The model is applied to a stylized version of the Italian electricity system and tested under varying levels of competition, market designs, and policy scenarios.
Score: 0.0
License: http://creativecommons.org/licenses/by-sa/4.0/
Abstract: Electricity systems are key to transforming today's society into a carbon-free economy. Long-term electricity market mechanisms, including auctions, support schemes, and other policy instruments, are critical in shaping the electricity generation mix. In light of the need for more advanced tools to support policymakers and other stakeholders in designing, testing, and evaluating long-term markets, this work presents a multi-agent reinforcement learning model capable of capturing the key features of decarbonizing energy systems. Profit-maximizing generation companies make investment decisions in the wholesale electricity market, responding to system needs, competitive dynamics, and policy signals. The model employs independent proximal policy optimization, which was selected for suitability to the decentralized and competitive environment. Nevertheless, given the inherent challenges of independent learning in multi-agent settings, an extensive hyperparameter search ensures that decentralized training yields market outcomes consistent with competitive behavior. The model is applied to a stylized version of the Italian electricity system and tested under varying levels of competition, market designs, and policy scenarios. Results highlight the critical role of market design for decarbonizing the electricity sector and avoiding price volatility. The proposed framework allows assessing long-term electricity markets in which multiple policy and market mechanisms interact simultaneously, with market participants responding and adapting to decarbonization pathways.

Related papers

Evaluating the Impact of Multiple DER Aggregators on Wholesale Energy Markets: A Hybrid Mean Field Approach [2.0535683313855055]
The integration of distributed energy resources into wholesale energy markets can greatly enhance grid flexibility, improve market efficiency, and contribute to a more sustainable energy future.<n>We study a wholesale market model featuring multiple DER aggregators, each controlling a portfolio of DER resources and bidding into the market on behalf of the DER asset owners.<n>We propose a reinforcement learning (RL)-based method to help each agent learn optimal strategies within the MFG framework, enhancing their ability to adapt to market conditions and uncertainties.
arXiv Detail & Related papers (2024-08-27T14:56:28Z)
A Machine Learning Framework to Deconstruct the Primary Drivers for Electricity Market Price Events [0.8192907805418581]
Power grids are moving towards 100% renewable energy source bulk power grids. Traditional root cause analysis and statistical approaches are rendered inapplicable to analyze and infer the main drivers behind price formation. We propose a machine learning-based analysis framework to deconstruct the primary drivers for price spike events in modern electricity markets with high renewable energy.
arXiv Detail & Related papers (2023-09-12T09:24:21Z)
Approximating Energy Market Clearing and Bidding With Model-Based Reinforcement Learning [0.0]
Multi-agent Reinforcement learning (MARL) is a promising new approach to predicting the expected profit-maximizing behavior of energy market participants in simulation. We provide a model of the energy market to a basic MARL algorithm in the form of a learned OPF approximation and explicit market rules. Our experiments demonstrate that the model reduces training time by about one order of magnitude but at the cost of a slightly worse performance.
arXiv Detail & Related papers (2023-03-03T08:26:22Z)
Machine learning applications for electricity market agent-based models: A systematic literature review [68.8204255655161]
Agent-based simulations are used to better understand the dynamics of the electricity market. Agent-based models provide the opportunity to integrate machine learning and artificial intelligence. We review 55 papers published between 2016 and 2021 which focus on machine learning applied to agent-based electricity market models.
arXiv Detail & Related papers (2022-06-05T14:52:26Z)
Modelling the transition to a low-carbon energy supply [91.3755431537592]
A transition to a low-carbon electricity supply is crucial to limit the impacts of climate change. Reducing carbon emissions could help prevent the world from reaching a tipping point, where runaway emissions are likely. Runaway emissions could lead to extremes in weather conditions around the world.
arXiv Detail & Related papers (2021-09-25T12:37:05Z)
Neural Fitted Q Iteration based Optimal Bidding Strategy in Real Time Reactive Power Market_1 [16.323822608442836]
In real time electricity markets, the objective of generation companies while bidding is to maximize their profit. Similar studies in reactive power markets have not been reported so far because the network voltage operating conditions have an increased impact on reactive power markets. The assumption of a suitable probability distribution function is unrealistic, making the strategies adopted in active power markets unsuitable for learning optimal bids in reactive power market mechanisms.
arXiv Detail & Related papers (2021-01-07T09:44:00Z)
Exploring market power using deep reinforcement learning for intelligent bidding strategies [69.3939291118954]
We find that capacity has an impact on the average electricity price in a single year. The value of $sim$25% and $sim$11% may vary between market structures and countries. We observe that the use of a market cap of approximately double the average market price has the effect of significantly decreasing this effect and maintaining a competitive market.
arXiv Detail & Related papers (2020-11-08T21:07:42Z)
A Multi-Agent Deep Reinforcement Learning Approach for a Distributed Energy Marketplace in Smart Grids [58.666456917115056]
This paper presents a Reinforcement Learning based energy market for a prosumer dominated microgrid. The proposed market model facilitates a real-time and demanddependent dynamic pricing environment, which reduces grid costs and improves the economic benefits for prosumers.
arXiv Detail & Related papers (2020-09-23T02:17:51Z)
A Deep Reinforcement Learning Framework for Continuous Intraday Market Bidding [69.37299910149981]
A key component for the successful renewable energy sources integration is the usage of energy storage. We propose a novel modelling framework for the strategic participation of energy storage in the European continuous intraday market. An distributed version of the fitted Q algorithm is chosen for solving this problem due to its sample efficiency. Results indicate that the agent converges to a policy that achieves in average higher total revenues than the benchmark strategy.
arXiv Detail & Related papers (2020-04-13T13:50:13Z)
Towards a Peer-to-Peer Energy Market: an Overview [68.8204255655161]
This work focuses on the electric power market, comparing the status quo with the recent trend towards the increase in distributed self-generation capabilities by prosumers. We introduce a potential multi-layered architecture for a Peer-to-Peer (P2P) energy market, discussing the fundamental aspects of local production and local consumption as part of a microgrid. To give a full picture to the reader, we also scrutinise relevant elements of energy trading, such as Smart Contract and grid stability.
arXiv Detail & Related papers (2020-03-02T20:32:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.