Scheduling and Communication Schemes for Decentralized Federated Learning

• URL: http://arxiv.org/abs/2311.16021v1
• Date: Mon, 27 Nov 2023 17:35:28 GMT
• Title: Scheduling and Communication Schemes for Decentralized Federated Learning
• Authors: Bahaa-Eldin Ali Abdelghany and Ana Fern\'andez-Vilas and Manuel Fern\'andez-Veiga and Nashwa El-Bendary and Ammar M. Hassan and Walid M. Abdelmoez
• Abstract summary: A decentralized federated learning (DFL) model with the gradient descent (SGD) algorithm has been introduced. Three scheduling policies for DFL have been proposed for communications between the clients and the parallel servers. Results show that the proposed scheduling polices have an impact both on the speed of convergence and in the final global model.
• Score: 0.31410859223862103
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated learning (FL) is a distributed machine learning paradigm in which a large number of clients coordinate with a central server to learn a model without sharing their own training data. One central server is not enough, due to problems of connectivity with clients. In this paper, a decentralized federated learning (DFL) model with the stochastic gradient descent (SGD) algorithm has been introduced, as a more scalable approach to improve the learning performance in a network of agents with arbitrary topology. Three scheduling policies for DFL have been proposed for communications between the clients and the parallel servers, and the convergence, accuracy, and loss have been tested in a totally decentralized mplementation of SGD. The experimental results show that the proposed scheduling polices have an impact both on the speed of convergence and in the final global model.

Related papers

• FedSPD: A Soft-clustering Approach for Personalized Decentralized Federated Learning [18.38030098837294]
  Federated learning is a framework for distributed clients to collaboratively train a machine learning model using local data. We propose FedSPD, an efficient personalized federated learning algorithm for the decentralized setting. We show that FedSPD learns accurate models even in low-connectivity networks.
  arXiv  Detail & Related papers  (2024-10-24T15:48:34Z)
• Communication Efficient ConFederated Learning: An Event-Triggered SAGA Approach [67.27031215756121]
  Federated learning (FL) is a machine learning paradigm that targets model training without gathering the local data over various data sources. Standard FL, which employs a single server, can only support a limited number of users, leading to degraded learning capability. In this work, we consider a multi-server FL framework, referred to as emphConfederated Learning (CFL) in order to accommodate a larger number of users.
  arXiv  Detail & Related papers  (2024-02-28T03:27:10Z)
• Decentralized Sporadic Federated Learning: A Unified Algorithmic Framework with Convergence Guarantees [18.24213566328972]
  Decentralized decentralized learning (DFL) captures FL settings where both (i) model updates and (ii) model aggregations are carried out by the clients without a central server. DSpodFL consistently achieves speeds compared with baselines under various system settings.
  arXiv  Detail & Related papers  (2024-02-05T19:02:19Z)
• FedLALR: Client-Specific Adaptive Learning Rates Achieve Linear Speedup for Non-IID Data [54.81695390763957]
  Federated learning is an emerging distributed machine learning method. We propose a heterogeneous local variant of AMSGrad, named FedLALR, in which each client adjusts its learning rate. We show that our client-specified auto-tuned learning rate scheduling can converge and achieve linear speedup with respect to the number of clients.
  arXiv  Detail & Related papers  (2023-09-18T12:35:05Z)
• Effectively Heterogeneous Federated Learning: A Pairing and Split Learning Based Approach [16.093068118849246]
  This paper presents a novel split federated learning (SFL) framework that pairs clients with different computational resources. A greedy algorithm is proposed by reconstructing the optimization of training latency as a graph edge selection problem. Simulation results show the proposed method can significantly improve the FL training speed and achieve high performance.
  arXiv  Detail & Related papers  (2023-08-26T11:10:54Z)
• Time-sensitive Learning for Heterogeneous Federated Edge Intelligence [52.83633954857744]
  We investigate real-time machine learning in a federated edge intelligence (FEI) system. FEI systems exhibit heterogenous communication and computational resource distribution. We propose a time-sensitive federated learning (TS-FL) framework to minimize the overall run-time for collaboratively training a shared ML model.
  arXiv  Detail & Related papers  (2023-01-26T08:13:22Z)
• DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training [84.81043932706375]
  We propose a novel personalized federated learning framework in a decentralized (peer-to-peer) communication protocol named Dis-PFL. Dis-PFL employs personalized sparse masks to customize sparse local models on the edge. We demonstrate that our method can easily adapt to heterogeneous local clients with varying computation complexities.
  arXiv  Detail & Related papers  (2022-06-01T02:20:57Z)
• Acceleration of Federated Learning with Alleviated Forgetting in Local Training [61.231021417674235]
  Federated learning (FL) enables distributed optimization of machine learning models while protecting privacy. We propose FedReg, an algorithm to accelerate FL with alleviated knowledge forgetting in the local training stage. Our experiments demonstrate that FedReg not only significantly improves the convergence rate of FL, especially when the neural network architecture is deep.
  arXiv  Detail & Related papers  (2022-03-05T02:31:32Z)
• Decentralized Federated Learning via Mutual Knowledge Transfer [37.5341683644709]
  Decentralized federated learning (DFL) is a problem in the Internet of things (IoT) systems. We propose a mutual knowledge transfer (Def-KT) algorithm where local clients fuse models by transferring their learnt knowledge to each other. Our experiments on the MNIST, Fashion-MNIST, and CIFAR10 datasets reveal datasets that the proposed Def-KT algorithm significantly outperforms the baseline DFL methods.
  arXiv  Detail & Related papers  (2020-12-24T01:43:53Z)
• Adaptive Serverless Learning [114.36410688552579]
  We propose a novel adaptive decentralized training approach, which can compute the learning rate from data dynamically. Our theoretical results reveal that the proposed algorithm can achieve linear speedup with respect to the number of workers. To reduce the communication-efficient overhead, we further propose a communication-efficient adaptive decentralized training approach.
  arXiv  Detail & Related papers  (2020-08-24T13:23:02Z)
• Coded Federated Learning [5.375775284252717]
  Federated learning is a method of training a global model from decentralized data distributed across client devices. Our results show that CFL allows the global model to converge nearly four times faster when compared to an uncoded approach.
  arXiv  Detail & Related papers  (2020-02-21T23:06:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.