Collaborative Policy Learning for Dynamic Scheduling Tasks in Cloud-Edge-Terminal IoT Networks Using Federated Reinforcement Learning

• URL: http://arxiv.org/abs/2307.00541v1
• Date: Sun, 2 Jul 2023 11:09:00 GMT
• Title: Collaborative Policy Learning for Dynamic Scheduling Tasks in Cloud-Edge-Terminal IoT Networks Using Federated Reinforcement Learning
• Authors: Do-Yup Kim, Da-Eun Lee, Ji-Wan Kim, Hyun-Suk Lee
• Abstract summary: We propose a novel collaborative policy learning framework for dynamic scheduling tasks. Our framework adaptively selects the tasks for collaborative learning in each round, taking into account the need for fairness among tasks.
• Score: 8.359770027722275
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we examine cloud-edge-terminal IoT networks, where edges undertake a range of typical dynamic scheduling tasks. In these IoT networks, a central policy for each task can be constructed at a cloud server. The central policy can be then used by the edges conducting the task, thereby mitigating the need for them to learn their own policy from scratch. Furthermore, this central policy can be collaboratively learned at the cloud server by aggregating local experiences from the edges, thanks to the hierarchical architecture of the IoT networks. To this end, we propose a novel collaborative policy learning framework for dynamic scheduling tasks using federated reinforcement learning. For effective learning, our framework adaptively selects the tasks for collaborative learning in each round, taking into account the need for fairness among tasks. In addition, as a key enabler of the framework, we propose an edge-agnostic policy structure that enables the aggregation of local policies from different edges. We then provide the convergence analysis of the framework. Through simulations, we demonstrate that our proposed framework significantly outperforms the approaches without collaborative policy learning. Notably, it accelerates the learning speed of the policies and allows newly arrived edges to adapt to their tasks more easily.

Related papers

• Structured Reinforcement Learning for Media Streaming at the Wireless Edge [15.742424623905825]
  Media streaming is the dominant application over wireless edge (access) networks. We develop and demonstrate learning-based policies for optimal decision making in a video streaming setting.
  arXiv  Detail & Related papers  (2024-04-10T19:25:51Z)
• What Planning Problems Can A Relational Neural Network Solve? [91.53684831950612]
  We present a circuit complexity analysis for relational neural networks representing policies for planning problems. We show that there are three general classes of planning problems, in terms of the growth of circuit width and depth. We also illustrate the utility of this analysis for designing neural networks for policy learning.
  arXiv  Detail & Related papers  (2023-12-06T18:47:28Z)
• Anomaly Detection for Scalable Task Grouping in Reinforcement Learning-based RAN Optimization [13.055378785343335]
  Training and maintaining learned models that work well across a large number of cell sites has become a pertinent problem. This paper proposes a scalable framework for constructing a reinforcement learning policy bank that can perform RAN optimization across a large number of cell sites.
  arXiv  Detail & Related papers  (2023-12-06T04:05:17Z)
• Planning to Practice: Efficient Online Fine-Tuning by Composing Goals in Latent Space [76.46113138484947]
  General-purpose robots require diverse repertoires of behaviors to complete challenging tasks in real-world unstructured environments. To address this issue, goal-conditioned reinforcement learning aims to acquire policies that can reach goals for a wide range of tasks on command. We propose Planning to Practice, a method that makes it practical to train goal-conditioned policies for long-horizon tasks.
  arXiv  Detail & Related papers  (2022-05-17T06:58:17Z)
• Policy Architectures for Compositional Generalization in Control [71.61675703776628]
  We introduce a framework for modeling entity-based compositional structure in tasks. Our policies are flexible and can be trained end-to-end without requiring any action primitives.
  arXiv  Detail & Related papers  (2022-03-10T06:44:24Z)
• Constructing a Good Behavior Basis for Transfer using Generalized Policy Updates [63.58053355357644]
  We study the problem of learning a good set of policies, so that when combined together, they can solve a wide variety of unseen reinforcement learning tasks. We show theoretically that having access to a specific set of diverse policies, which we call a set of independent policies, can allow for instantaneously achieving high-level performance.
  arXiv  Detail & Related papers  (2021-12-30T12:20:46Z)
• Towards Exploiting Geometry and Time for FastOff-Distribution Adaptation in Multi-Task RobotLearning [17.903462188570067]
  We train policies for a base set of pre-training tasks, then experiment with adapting to new off-distribution tasks. We find that combining low-complexity target policy classes, base policies as black-box priors, and simple optimization algorithms allows us to acquire new tasks outside the base task distribution.
  arXiv  Detail & Related papers  (2021-06-24T02:13:50Z)
• Learning Adaptive Exploration Strategies in Dynamic Environments Through Informed Policy Regularization [100.72335252255989]
  We study the problem of learning exploration-exploitation strategies that effectively adapt to dynamic environments. We propose a novel algorithm that regularizes the training of an RNN-based policy using informed policies trained to maximize the reward in each task.
  arXiv  Detail & Related papers  (2020-05-06T16:14:48Z)
• Decentralized MCTS via Learned Teammate Models [89.24858306636816]
  We present a trainable online decentralized planning algorithm based on decentralized Monte Carlo Tree Search. We show that deep learning and convolutional neural networks can be employed to produce accurate policy approximators.
  arXiv  Detail & Related papers  (2020-03-19T13:10:20Z)
• Real-Time Edge Intelligence in the Making: A Collaborative Learning Framework via Federated Meta-Learning [24.00507627945666]
  IoT applications at the network edge demand intelligent decisions in a real-time manner. We propose a platform-aided collaborative learning framework where a model is first trained across a set of source edge nodes. We then adapt the model to learn a new task at the target edge node, using a few samples only.
  arXiv  Detail & Related papers  (2020-01-09T21:37:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.