Federated Multi-Agent Deep Reinforcement Learning Approach via
Physics-Informed Reward for Multi-Microgrid Energy Management
- URL: http://arxiv.org/abs/2301.00641v1
- Date: Thu, 29 Dec 2022 08:35:11 GMT
- Title: Federated Multi-Agent Deep Reinforcement Learning Approach via
Physics-Informed Reward for Multi-Microgrid Energy Management
- Authors: Yuanzheng Li, Shangyang He, Yang Li, Yang Shi, Zhigang Zeng
- Abstract summary: This paper proposes a federated multi-agent deep reinforcement learning (F-MADRL) algorithm via the physics-informed reward.
In this algorithm, the federated learning mechanism is introduced to train the F-MADRL algorithm thus ensures the privacy and the security of data.
Experiments are conducted on Oak Ridge national laboratory distributed energy control communication lab microgrid (ORNL-MG) test system.
- Score: 34.18923657108073
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The utilization of large-scale distributed renewable energy promotes the
development of the multi-microgrid (MMG), which raises the need of developing
an effective energy management method to minimize economic costs and keep self
energy-sufficiency. The multi-agent deep reinforcement learning (MADRL) has
been widely used for the energy management problem because of its real-time
scheduling ability. However, its training requires massive energy operation
data of microgrids (MGs), while gathering these data from different MGs would
threaten their privacy and data security. Therefore, this paper tackles this
practical yet challenging issue by proposing a federated multi-agent deep
reinforcement learning (F-MADRL) algorithm via the physics-informed reward. In
this algorithm, the federated learning (FL) mechanism is introduced to train
the F-MADRL algorithm thus ensures the privacy and the security of data. In
addition, a decentralized MMG model is built, and the energy of each
participated MG is managed by an agent, which aims to minimize economic costs
and keep self energy-sufficiency according to the physics-informed reward. At
first, MGs individually execute the self-training based on local energy
operation data to train their local agent models. Then, these local models are
periodically uploaded to a server and their parameters are aggregated to build
a global agent, which will be broadcasted to MGs and replace their local
agents. In this way, the experience of each MG agent can be shared and the
energy operation data is not explicitly transmitted, thus protecting the
privacy and ensuring data security. Finally, experiments are conducted on Oak
Ridge national laboratory distributed energy control communication lab
microgrid (ORNL-MG) test system, and the comparisons are carried out to verify
the effectiveness of introducing the FL mechanism and the outperformance of our
proposed F-MADRL.
Related papers
- Collaborative Optimization of Multi-microgrids System with Shared Energy
Storage Based on Multi-agent Stochastic Game and Reinforcement Learning [8.511196076836592]
The proposed MMG system framework can reduce energy fluctuations in the main grid by 1746.5kW in 24 hours and achieve a cost reduction of 16.21% in the test.
The superiority of the proposed algorithms is verified through their fast convergence speed and excellent optimization performance.
arXiv Detail & Related papers (2023-06-19T07:55:41Z) - Multi-Microgrid Collaborative Optimization Scheduling Using an Improved
Multi-Agent Soft Actor-Critic Algorithm [8.461537684562776]
A multi-microgrid (MMG) system consists of multiple renewable energy microgrids belonging to different operating entities.
This paper proposes a MMG collaborative optimization scheduling model based on a multi-agent centralized training distributed execution framework.
arXiv Detail & Related papers (2023-04-01T22:44:52Z) - Distributed Energy Management and Demand Response in Smart Grids: A
Multi-Agent Deep Reinforcement Learning Framework [53.97223237572147]
This paper presents a multi-agent Deep Reinforcement Learning (DRL) framework for autonomous control and integration of renewable energy resources into smart power grid systems.
In particular, the proposed framework jointly considers demand response (DR) and distributed energy management (DEM) for residential end-users.
arXiv Detail & Related papers (2022-11-29T01:18:58Z) - Multi-agent Bayesian Deep Reinforcement Learning for Microgrid Energy
Management under Communication Failures [10.099371194251052]
We propose a multi-agent Bayesian deep reinforcement learning (BA-DRL) method for MG energy management under communication failures.
BA-DRL has 4.1% and 10.3% higher reward than Nash Deep Q-learning (Nash-DQN) and alternating direction method of multipliers (ADMM) respectively under 1% communication failure probability.
arXiv Detail & Related papers (2021-11-22T03:08:10Z) - Deep Reinforcement Learning Based Multidimensional Resource Management
for Energy Harvesting Cognitive NOMA Communications [64.1076645382049]
Combination of energy harvesting (EH), cognitive radio (CR), and non-orthogonal multiple access (NOMA) is a promising solution to improve energy efficiency.
In this paper, we study the spectrum, energy, and time resource management for deterministic-CR-NOMA IoT systems.
arXiv Detail & Related papers (2021-09-17T08:55:48Z) - Dif-MAML: Decentralized Multi-Agent Meta-Learning [54.39661018886268]
We propose a cooperative multi-agent meta-learning algorithm, referred to as MAML or Dif-MAML.
We show that the proposed strategy allows a collection of agents to attain agreement at a linear rate and to converge to a stationary point of the aggregate MAML.
Simulation results illustrate the theoretical findings and the superior performance relative to the traditional non-cooperative setting.
arXiv Detail & Related papers (2020-10-06T16:51:09Z) - Risk-Aware Energy Scheduling for Edge Computing with Microgrid: A
Multi-Agent Deep Reinforcement Learning Approach [82.6692222294594]
We study a risk-aware energy scheduling problem for a microgrid-powered MEC network.
We derive the solution by applying a multi-agent deep reinforcement learning (MADRL)-based advantage actor-critic (A3C) algorithm with shared neural networks.
arXiv Detail & Related papers (2020-02-21T02:14:38Z) - Multi-Agent Meta-Reinforcement Learning for Self-Powered and Sustainable
Edge Computing Systems [87.4519172058185]
An effective energy dispatch mechanism for self-powered wireless networks with edge computing capabilities is studied.
A novel multi-agent meta-reinforcement learning (MAMRL) framework is proposed to solve the formulated problem.
Experimental results show that the proposed MAMRL model can reduce up to 11% non-renewable energy usage and by 22.4% the energy cost.
arXiv Detail & Related papers (2020-02-20T04:58:07Z) - Dynamic Energy Dispatch Based on Deep Reinforcement Learning in
IoT-Driven Smart Isolated Microgrids [8.623472323825556]
Microgrids (MGs) are small, local power grids that can operate independently from the larger utility grid.
This paper focuses on deep reinforcement learning (DRL)-based energy dispatch for IoT-driven smart isolated MGs.
Two novel DRL algorithms are proposed to derive energy dispatch policies with and without fully observable state information.
arXiv Detail & Related papers (2020-02-07T01:44:18Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.