Multi-Agent Meta-Reinforcement Learning for Self-Powered and Sustainable Edge Computing Systems

• URL: http://arxiv.org/abs/2002.08567v3
• Date: Wed, 10 Feb 2021 02:47:56 GMT
• Title: Multi-Agent Meta-Reinforcement Learning for Self-Powered and Sustainable Edge Computing Systems
• Authors: Md. Shirajum Munir, Nguyen H. Tran, Walid Saad, Choong Seon Hong
• Abstract summary: An effective energy dispatch mechanism for self-powered wireless networks with edge computing capabilities is studied. A novel multi-agent meta-reinforcement learning (MAMRL) framework is proposed to solve the formulated problem. Experimental results show that the proposed MAMRL model can reduce up to 11% non-renewable energy usage and by 22.4% the energy cost.
• Score: 87.4519172058185
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The stringent requirements of mobile edge computing (MEC) applications and functions fathom the high capacity and dense deployment of MEC hosts to the upcoming wireless networks. However, operating such high capacity MEC hosts can significantly increase energy consumption. Thus, a base station (BS) unit can act as a self-powered BS. In this paper, an effective energy dispatch mechanism for self-powered wireless networks with edge computing capabilities is studied. First, a two-stage linear stochastic programming problem is formulated with the goal of minimizing the total energy consumption cost of the system while fulfilling the energy demand. Second, a semi-distributed data-driven solution is proposed by developing a novel multi-agent meta-reinforcement learning (MAMRL) framework to solve the formulated problem. In particular, each BS plays the role of a local agent that explores a Markovian behavior for both energy consumption and generation while each BS transfers time-varying features to a meta-agent. Sequentially, the meta-agent optimizes (i.e., exploits) the energy dispatch decision by accepting only the observations from each local agent with its own state information. Meanwhile, each BS agent estimates its own energy dispatch policy by applying the learned parameters from meta-agent. Finally, the proposed MAMRL framework is benchmarked by analyzing deterministic, asymmetric, and stochastic environments in terms of non-renewable energy usages, energy cost, and accuracy. Experimental results show that the proposed MAMRL model can reduce up to 11% non-renewable energy usage and by 22.4% the energy cost (with 95.8% prediction accuracy), compared to other baseline methods.

Related papers

• Multiagent Reinforcement Learning with an Attention Mechanism for Improving Energy Efficiency in LoRa Networks [52.96907334080273]
  As the network scale increases, the energy efficiency of LoRa networks decreases sharply due to severe packet collisions. We propose a transmission parameter allocation algorithm based on multiagent reinforcement learning (MALoRa) Simulation results demonstrate that MALoRa significantly improves the system EE compared with baseline algorithms.
  arXiv  Detail & Related papers  (2023-09-16T11:37:23Z)
• Collaborative Optimization of Multi-microgrids System with Shared Energy Storage Based on Multi-agent Stochastic Game and Reinforcement Learning [8.511196076836592]
  The proposed MMG system framework can reduce energy fluctuations in the main grid by 1746.5kW in 24 hours and achieve a cost reduction of 16.21% in the test. The superiority of the proposed algorithms is verified through their fast convergence speed and excellent optimization performance.
  arXiv  Detail & Related papers  (2023-06-19T07:55:41Z)
• Federated Multi-Agent Deep Reinforcement Learning Approach via Physics-Informed Reward for Multi-Microgrid Energy Management [34.18923657108073]
  This paper proposes a federated multi-agent deep reinforcement learning (F-MADRL) algorithm via the physics-informed reward. In this algorithm, the federated learning mechanism is introduced to train the F-MADRL algorithm thus ensures the privacy and the security of data. Experiments are conducted on Oak Ridge national laboratory distributed energy control communication lab microgrid (ORNL-MG) test system.
  arXiv  Detail & Related papers  (2022-12-29T08:35:11Z)
• Deep Reinforcement Learning Based Multidimensional Resource Management for Energy Harvesting Cognitive NOMA Communications [64.1076645382049]
  Combination of energy harvesting (EH), cognitive radio (CR), and non-orthogonal multiple access (NOMA) is a promising solution to improve energy efficiency. In this paper, we study the spectrum, energy, and time resource management for deterministic-CR-NOMA IoT systems.
  arXiv  Detail & Related papers  (2021-09-17T08:55:48Z)
• Energy-Efficient and Federated Meta-Learning via Projected Stochastic Gradient Ascent [79.58680275615752]
  We propose an energy-efficient federated meta-learning framework. We assume each task is owned by a separate agent, so a limited number of tasks is used to train a meta-model.
  arXiv  Detail & Related papers  (2021-05-31T08:15:44Z)
• Demand-Side Scheduling Based on Multi-Agent Deep Actor-Critic Learning for Smart Grids [56.35173057183362]
  We consider the problem of demand-side energy management, where each household is equipped with a smart meter that is able to schedule home appliances online. The goal is to minimize the overall cost under a real-time pricing scheme. We propose the formulation of a smart grid environment as a Markov game.
  arXiv  Detail & Related papers  (2020-05-05T07:32:40Z)
• Risk-Aware Energy Scheduling for Edge Computing with Microgrid: A Multi-Agent Deep Reinforcement Learning Approach [82.6692222294594]
  We study a risk-aware energy scheduling problem for a microgrid-powered MEC network. We derive the solution by applying a multi-agent deep reinforcement learning (MADRL)-based advantage actor-critic (A3C) algorithm with shared neural networks.
  arXiv  Detail & Related papers  (2020-02-21T02:14:38Z)
• Dynamic Energy Dispatch Based on Deep Reinforcement Learning in IoT-Driven Smart Isolated Microgrids [8.623472323825556]
  Microgrids (MGs) are small, local power grids that can operate independently from the larger utility grid. This paper focuses on deep reinforcement learning (DRL)-based energy dispatch for IoT-driven smart isolated MGs. Two novel DRL algorithms are proposed to derive energy dispatch policies with and without fully observable state information.
  arXiv  Detail & Related papers  (2020-02-07T01:44:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.