Online Service Migration in Edge Computing with Incomplete Information: A Deep Recurrent Actor-Critic Method

• URL: http://arxiv.org/abs/2012.08679v3
• Date: Sun, 11 Apr 2021 10:53:50 GMT
• Title: Online Service Migration in Edge Computing with Incomplete Information: A Deep Recurrent Actor-Critic Method
• Authors: Jin Wang, Jia Hu, and Geyong Min
• Abstract summary: Multi-access Edge Computing (MEC) is an emerging computing paradigm that extends cloud computing to the network edge. Service migration needs to decide where to migrate user services for maintaining high Quality-of-Service (QoS) We propose a new learning-driven method, namely Deep Recurrent ActorCritic based service Migration (DRACM), which is usercentric and can make effective online migration decisions.
• Score: 18.891775769665102
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-access Edge Computing (MEC) is an emerging computing paradigm that extends cloud computing to the network edge (e.g., base stations, MEC servers) to support resource-intensive applications on mobile devices. As a crucial problem in MEC, service migration needs to decide where to migrate user services for maintaining high Quality-of-Service (QoS), when users roam between MEC servers with limited coverage and capacity. However, finding an optimal migration policy is intractable due to the highly dynamic MEC environment and user mobility. Many existing works make centralized migration decisions based on complete system-level information, which can be time-consuming and suffer from the scalability issue with the rapidly increasing number of mobile users. To address these challenges, we propose a new learning-driven method, namely Deep Recurrent Actor-Critic based service Migration (DRACM), which is user-centric and can make effective online migration decisions given incomplete system-level information. Specifically, the service migration problem is modeled as a Partially Observable Markov Decision Process (POMDP). To solve the POMDP, we design an encoder network that combines a Long Short-Term Memory (LSTM) and an embedding matrix for effective extraction of hidden information. We then propose a tailored off-policy actor-critic algorithm with a clipped surrogate objective for efficient training. Results from extensive experiments based on real-world mobility traces demonstrate that our method consistently outperforms both the heuristic and state-of-the-art learning-driven algorithms, and achieves near-optimal results on various MEC scenarios.

Related papers

• Federated Prompt-based Decision Transformer for Customized VR Services in Mobile Edge Computing System [9.269074750399657]
  We first introduce a quality of experience (QoE) metric to measure user experience. Then, a QoE problem is formulated for resource allocation to ensure the highest possible user experience. We propose a framework that employs federated learning (FL) and prompt-based sequence modeling to pre-train a common model.
  arXiv  Detail & Related papers  (2024-02-15T05:56:35Z)
• Edge computing service deployment and task offloading based on multi-task high-dimensional multi-objective optimization [5.64850919046892]
  This study investigates service deployment and task offloading challenges in a multi-user environment. To ensure stable service provisioning, beyond considering latency, energy consumption, and cost, network reliability is also incorporated. To promote equitable usage of edge servers, load balancing is introduced as a fourth task offloading objective.
  arXiv  Detail & Related papers  (2023-12-07T07:30:47Z)
• Age-Based Scheduling for Mobile Edge Computing: A Deep Reinforcement Learning Approach [58.911515417156174]
  We propose a new definition of Age of Information (AoI) and, based on the redefined AoI, we formulate an online AoI problem for MEC systems. We introduce Post-Decision States (PDSs) to exploit the partial knowledge of the system's dynamics. We also combine PDSs with deep RL to further improve the algorithm's applicability, scalability, and robustness.
  arXiv  Detail & Related papers  (2023-12-01T01:30:49Z)
• Task Graph offloading via Deep Reinforcement Learning in Mobile Edge Computing [6.872434270841794]
  This paper investigates the task graph offloading in MEC, considering the time-varying capabilities of edge computing devices. To adapt to environmental changes, we model the task graph scheduling for computation offloading as a Markov Decision Process. Then, we design a deep reinforcement learning algorithm (SATA-DRL) to learn the task scheduling strategy from the interaction with the environment.
  arXiv  Detail & Related papers  (2023-09-19T12:26:56Z)
• A Multi-Head Ensemble Multi-Task Learning Approach for Dynamical Computation Offloading [62.34538208323411]
  We propose a multi-head ensemble multi-task learning (MEMTL) approach with a shared backbone and multiple prediction heads (PHs) MEMTL outperforms benchmark methods in both the inference accuracy and mean square error without requiring additional training data.
  arXiv  Detail & Related papers  (2023-09-02T11:01:16Z)
• Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence [76.96698721128406]
  Mobile edge computing (MEC) considered a novel paradigm for computation and delay-sensitive tasks in fifth generation (5G) networks and beyond. This paper provides a comprehensive research review on free-enabled RL and offers insight for development.
  arXiv  Detail & Related papers  (2022-01-27T10:02:54Z)
• Distributed Reinforcement Learning for Privacy-Preserving Dynamic Edge Caching [91.50631418179331]
  A privacy-preserving distributed deep policy gradient (P2D3PG) is proposed to maximize the cache hit rates of devices in the MEC networks. We convert the distributed optimizations into model-free Markov decision process problems and then introduce a privacy-preserving federated learning method for popularity prediction.
  arXiv  Detail & Related papers  (2021-10-20T02:48:27Z)
• Toward Multiple Federated Learning Services Resource Sharing in Mobile Edge Networks [88.15736037284408]
  We study a new model of multiple federated learning services at the multi-access edge computing server. We propose a joint resource optimization and hyper-learning rate control problem, namely MS-FEDL. Our simulation results demonstrate the convergence performance of our proposed algorithms.
  arXiv  Detail & Related papers  (2020-11-25T01:29:41Z)
• Dif-MAML: Decentralized Multi-Agent Meta-Learning [54.39661018886268]
  We propose a cooperative multi-agent meta-learning algorithm, referred to as MAML or Dif-MAML. We show that the proposed strategy allows a collection of agents to attain agreement at a linear rate and to converge to a stationary point of the aggregate MAML. Simulation results illustrate the theoretical findings and the superior performance relative to the traditional non-cooperative setting.
  arXiv  Detail & Related papers  (2020-10-06T16:51:09Z)
• Fast Adaptive Task Offloading in Edge Computing based on Meta Reinforcement Learning [44.81038225683222]
  Multi-access edge computing (MEC) aims to extend cloud service to the network edge to reduce network traffic and service latency. A fundamental problem in MEC is how to efficiently offload heterogeneous tasks of mobile applications from user equipment (UE) to MEC hosts. We propose a task offloading method based on meta reinforcement learning, which can adapt fast to new environments.
  arXiv  Detail & Related papers  (2020-08-05T10:16:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.