A Hierarchical MARL-Based Approach for Coordinated Retail P2P Trading and Wholesale Market Participation of DERs

• URL: http://arxiv.org/abs/2604.20586v1
• Date: Wed, 22 Apr 2026 14:02:56 GMT
• Title: A Hierarchical MARL-Based Approach for Coordinated Retail P2P Trading and Wholesale Market Participation of DERs
• Abstract summary: This work proposes a market engagement framework that leverages a hierarchical multi-agent deep reinforcement learning (MARL) approach to enable individual prosumers to participate in peer-to-peer retail auctions.<n>Ultimately, a Stackelberg game is proposed to coordinate this hierarchical MARL-based DER market participation framework toward enhanced market performance.
• Score: 3.108481950101193
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The ongoing shift towards decentralization of the electric energy sector, driven by the growing electrification across end-use sectors, and widespread adoption of distributed energy resources (DERs), necessitates their active participation in the electricity markets to support grid operations. Furthermore, with bi-directional energy and communication flows becoming standard, intelligent, easy-to-deploy, resource-conservative demand-side participation is expected to play a critical role in securing power grid operational flexibility and market efficiency. This work proposes a market engagement framework that leverages a hierarchical multi-agent deep reinforcement learning (MARL) approach to enable individual prosumers to participate in peer-to-peer retail auctions and further aggregate these intelligent prosumers to facilitate effective DER participation in wholesale markets. Ultimately, a Stackelberg game is proposed to coordinate this hierarchical MARL-based DER market participation framework toward enhanced market performance.

Related papers

• Diffusing to Coordinate: Efficient Online Multi-Agent Diffusion Policies [51.24079409973799]
  Diffusion-based generative models are well-positioned to meet the needs of online Multi-Agent Reinforcement Learning (MARL)<n>We propose among the first underlineOnline off-policy underlineMARL framework using underlineDiffusion policies to orchestrate coordination.<n>Our key innovation is a relaxed policy objective that maximizes scaled joint entropy, facilitating effective exploration without relying on tractable likelihood.
  arXiv  Detail & Related papers  (2026-02-20T15:38:02Z)
• Scalable Fairness Shaping with LLM-Guided Multi-Agent Reinforcement Learning for Peer-to-Peer Electricity Markets [3.7321070110102075]
  Peer-to-peer (P2P) energy trading is becoming central to modern distribution systems.<n>A fairness-aware multiagent reinforcement learning framework, FairMarket-RL, is proposed.<n>The framework shifts exchanges toward local P2P trades, lowers consumer costs relative to grid-only procurement, sustains strong fairness across participants, and preserves utility viability.
  arXiv  Detail & Related papers  (2025-08-26T02:25:17Z)
• VAE-GAN Based Price Manipulation in Coordinated Local Energy Markets [3.498661956610689]
  This paper introduces a model for coordinating prosumers with heterogeneous distributed energy resources (DERs) in a local energy market (LEM)<n>The proposed LEM scheme utilizes a data-driven, model-free reinforcement learning approach based on the multi-agent deep deterministic policy gradient (MADDPG)<n>We investigate a price manipulation strategy using a variational auto encoder-generative adversarial network (VAE-GAN) model, which allows utilities to adjust price signals in a way that induces financial losses for the prosumers.
  arXiv  Detail & Related papers  (2025-07-26T07:38:27Z)
• Evaluating the Impact of Multiple DER Aggregators on Wholesale Energy Markets: A Hybrid Mean Field Approach [2.0535683313855055]
  The integration of distributed energy resources into wholesale energy markets can greatly enhance grid flexibility, improve market efficiency, and contribute to a more sustainable energy future.<n>We study a wholesale market model featuring multiple DER aggregators, each controlling a portfolio of DER resources and bidding into the market on behalf of the DER asset owners.<n>We propose a reinforcement learning (RL)-based method to help each agent learn optimal strategies within the MFG framework, enhancing their ability to adapt to market conditions and uncertainties.
  arXiv  Detail & Related papers  (2024-08-27T14:56:28Z)
• Temporal-Aware Deep Reinforcement Learning for Energy Storage Bidding in Energy and Contingency Reserve Markets [13.03742132147551]
  We develop a novel BESS joint bidding strategy that utilizes deep reinforcement learning (DRL) to bid in the spot and contingency frequency control ancillary services markets. Unlike conventional "black-box" DRL model, our approach is more interpretable and provides valuable insights into the temporal bidding behavior of BESS.
  arXiv  Detail & Related papers  (2024-02-29T12:41:54Z)
• Proximal Policy Optimization Based Reinforcement Learning for Joint Bidding in Energy and Frequency Regulation Markets [6.175137568373435]
  Energy arbitrage can be a significant source of revenue for the battery energy storage system (BESS) It is crucial for the BESS to carefully decide how much capacity to assign to each market to maximize the total profit under uncertain market conditions. This paper formulates the bidding problem of the BESS as a Markov Decision Process, which enables the BESS to participate in both the spot market and the FCAS market to maximize profit.
  arXiv  Detail & Related papers  (2022-12-13T13:07:31Z)
• Distributed Energy Management and Demand Response in Smart Grids: A Multi-Agent Deep Reinforcement Learning Framework [53.97223237572147]
  This paper presents a multi-agent Deep Reinforcement Learning (DRL) framework for autonomous control and integration of renewable energy resources into smart power grid systems. In particular, the proposed framework jointly considers demand response (DR) and distributed energy management (DEM) for residential end-users.
  arXiv  Detail & Related papers  (2022-11-29T01:18:58Z)
• Roles of Retailers in the Peer-to-Peer Electricity Market: A Single Retailer Perspective [60.293347226071795]
  This perspective takes the first step to discuss the feasibility of retailers' involvement in the peer-to-peer market. We identify key characteristics of retail-based and peer-to-peer electricity markets and discuss our viewpoint on how to incorporate a single retailer in a peer-to-peer market.
  arXiv  Detail & Related papers  (2021-10-12T00:02:08Z)
• A Multi-Agent Deep Reinforcement Learning Approach for a Distributed Energy Marketplace in Smart Grids [58.666456917115056]
  This paper presents a Reinforcement Learning based energy market for a prosumer dominated microgrid. The proposed market model facilitates a real-time and demanddependent dynamic pricing environment, which reduces grid costs and improves the economic benefits for prosumers.
  arXiv  Detail & Related papers  (2020-09-23T02:17:51Z)
• A Deep Reinforcement Learning Framework for Continuous Intraday Market Bidding [69.37299910149981]
  A key component for the successful renewable energy sources integration is the usage of energy storage. We propose a novel modelling framework for the strategic participation of energy storage in the European continuous intraday market. An distributed version of the fitted Q algorithm is chosen for solving this problem due to its sample efficiency. Results indicate that the agent converges to a policy that achieves in average higher total revenues than the benchmark strategy.
  arXiv  Detail & Related papers  (2020-04-13T13:50:13Z)
• Towards a Peer-to-Peer Energy Market: an Overview [68.8204255655161]
  This work focuses on the electric power market, comparing the status quo with the recent trend towards the increase in distributed self-generation capabilities by prosumers. We introduce a potential multi-layered architecture for a Peer-to-Peer (P2P) energy market, discussing the fundamental aspects of local production and local consumption as part of a microgrid. To give a full picture to the reader, we also scrutinise relevant elements of energy trading, such as Smart Contract and grid stability.
  arXiv  Detail & Related papers  (2020-03-02T20:32:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.