On Decentralizing Federated Reinforcement Learning in Multi-Robot Scenarios

• URL: http://arxiv.org/abs/2207.09372v1
• Date: Tue, 19 Jul 2022 16:17:12 GMT
• Title: On Decentralizing Federated Reinforcement Learning in Multi-Robot Scenarios
• Authors: Jayprakash S. Nair, Divya D. Kulkarni, Ajitem Joshi, Sruthy Suresh
• Abstract summary: Federated Learning (FL) allows for collaboratively aggregating learned information across several computing devices. Mobile agents can perform the task of Decentralized Federated Reinforcement Learning (dFRL) Results obtained from experiments carried out using Q-learning and SARSA by aggregating their corresponding Q-tables.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated Learning (FL) allows for collaboratively aggregating learned information across several computing devices and sharing the same amongst them, thereby tackling issues of privacy and the need of huge bandwidth. FL techniques generally use a central server or cloud for aggregating the models received from the devices. Such centralized FL techniques suffer from inherent problems such as failure of the central node and bottlenecks in channel bandwidth. When FL is used in conjunction with connected robots serving as devices, a failure of the central controlling entity can lead to a chaotic situation. This paper describes a mobile agent based paradigm to decentralize FL in multi-robot scenarios. Using Webots, a popular free open-source robot simulator, and Tartarus, a mobile agent platform, we present a methodology to decentralize federated learning in a set of connected robots. With Webots running on different connected computing systems, we show how mobile agents can perform the task of Decentralized Federated Reinforcement Learning (dFRL). Results obtained from experiments carried out using Q-learning and SARSA by aggregating their corresponding Q-tables, show the viability of using decentralized FL in the domain of robotics. Since the proposed work can be used in conjunction with other learning algorithms and also real robots, it can act as a vital tool for the study of decentralized FL using heterogeneous learning algorithms concurrently in multi-robot scenarios.

Related papers

• Fed-EC: Bandwidth-Efficient Clustering-Based Federated Learning For Autonomous Visual Robot Navigation [7.8839937556789375]
  Federated-EmbedCluster (Fed-EC) is a clustering-based federated learning framework deployed with vision based autonomous robot navigation in diverse outdoor environments. Fed-EC reduces the communication size by 23x for each robot while matching the performance of centralized learning for goal-oriented navigation and outperforms local learning.
  arXiv  Detail & Related papers  (2024-11-06T18:44:09Z)
• Communication Efficient ConFederated Learning: An Event-Triggered SAGA Approach [67.27031215756121]
  Federated learning (FL) is a machine learning paradigm that targets model training without gathering the local data over various data sources. Standard FL, which employs a single server, can only support a limited number of users, leading to degraded learning capability. In this work, we consider a multi-server FL framework, referred to as emphConfederated Learning (CFL) in order to accommodate a larger number of users.
  arXiv  Detail & Related papers  (2024-02-28T03:27:10Z)
• LPAC: Learnable Perception-Action-Communication Loops with Applications to Coverage Control [80.86089324742024]
  We propose a learnable Perception-Action-Communication (LPAC) architecture for the problem. CNN processes localized perception; a graph neural network (GNN) facilitates robot communications. Evaluations show that the LPAC models outperform standard decentralized and centralized coverage control algorithms.
  arXiv  Detail & Related papers  (2024-01-10T00:08:00Z)
• Coordination-free Decentralised Federated Learning on Complex Networks: Overcoming Heterogeneity [2.6849848612544]
  Federated Learning (FL) is a framework for performing a learning task in an edge computing scenario. We propose a communication-efficient Decentralised Federated Learning (DFL) algorithm able to cope with them. Our solution allows devices communicating only with their direct neighbours to train an accurate model.
  arXiv  Detail & Related papers  (2023-12-07T18:24:19Z)
• Robot Fleet Learning via Policy Merging [58.5086287737653]
  We propose FLEET-MERGE to efficiently merge policies in the fleet setting. We show that FLEET-MERGE consolidates the behavior of policies trained on 50 tasks in the Meta-World environment. We introduce a novel robotic tool-use benchmark, FLEET-TOOLS, for fleet policy learning in compositional and contact-rich robot manipulation tasks.
  arXiv  Detail & Related papers  (2023-10-02T17:23:51Z)
• Fedstellar: A Platform for Decentralized Federated Learning [10.014744081331672]
  In 2016, Google proposed Federated Learning (FL) as a novel paradigm to train Machine Learning (ML) models across the participants of a federation. This paper presents Fedstellar, a platform designed to train FL models in a decentralized, semi-decentralized, and centralized fashion across diverse federations.
  arXiv  Detail & Related papers  (2023-06-16T10:34:49Z)
• Confederated Learning: Federated Learning with Decentralized Edge Servers [42.766372620288585]
  Federated learning (FL) is an emerging machine learning paradigm that allows to accomplish model training without aggregating data at a central server. We propose a ConFederated Learning (CFL) framework, in which each server is connected with an individual set of devices. The proposed algorithm employs a random scheduling policy which randomly selects a subset of devices to access their respective servers at each iteration.
  arXiv  Detail & Related papers  (2022-05-30T07:56:58Z)
• Learning Connectivity for Data Distribution in Robot Teams [96.39864514115136]
  We propose a task-agnostic, decentralized, low-latency method for data distribution in ad-hoc networks using Graph Neural Networks (GNN) Our approach enables multi-agent algorithms based on global state information to function by ensuring it is available at each robot. We train the distributed GNN communication policies via reinforcement learning using the average Age of Information as the reward function and show that it improves training stability compared to task-specific reward functions.
  arXiv  Detail & Related papers  (2021-03-08T21:48:55Z)
• Opportunities of Federated Learning in Connected, Cooperative and Automated Industrial Systems [44.627847349764664]
  Next-generation industrial systems have driven advances in ultra-reliable, low latency communications. Distributed machine learning (FL) represents a mushrooming multidisciplinary research area weaving in sensing, communication and learning. This article explores emerging opportunities of FL for the next-generation networked industrial systems.
  arXiv  Detail & Related papers  (2021-01-09T14:27:52Z)
• From Federated to Fog Learning: Distributed Machine Learning over Heterogeneous Wireless Networks [71.23327876898816]
  Federated learning has emerged as a technique for training ML models at the network edge by leveraging processing capabilities across the nodes that collect the data. We advocate a new learning paradigm called fog learning which will intelligently distribute ML model training across the continuum of nodes from edge devices to cloud servers.
  arXiv  Detail & Related papers  (2020-06-07T05:11:18Z)
• Wireless Communications for Collaborative Federated Learning [160.82696473996566]
  Internet of Things (IoT) devices may not be able to transmit their collected data to a central controller for training machine learning models. Google's seminal FL algorithm requires all devices to be directly connected with a central controller. This paper introduces a novel FL framework, called collaborative FL (CFL), which enables edge devices to implement FL with less reliance on a central controller.
  arXiv  Detail & Related papers  (2020-06-03T20:00:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.