AEDFL: Efficient Asynchronous Decentralized Federated Learning with Heterogeneous Devices

• URL: http://arxiv.org/abs/2312.10935v1
• Date: Mon, 18 Dec 2023 05:18:17 GMT
• Title: AEDFL: Efficient Asynchronous Decentralized Federated Learning with Heterogeneous Devices
• Authors: Ji Liu and Tianshi Che and Yang Zhou and Ruoming Jin and Huaiyu Dai and Dejing Dou and Patrick Valduriez
• Abstract summary: We propose an Asynchronous Efficient Decentralized FL framework, i.e., AEDFL, in heterogeneous environments. First, we propose an asynchronous FL system model with an efficient model aggregation method for improving the FL convergence. Second, we propose a dynamic staleness-aware model update approach to achieve superior accuracy. Third, we propose an adaptive sparse training method to reduce communication and computation costs without significant accuracy degradation.
• Score: 61.66943750584406
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Federated Learning (FL) has achieved significant achievements recently, enabling collaborative model training on distributed data over edge devices. Iterative gradient or model exchanges between devices and the centralized server in the standard FL paradigm suffer from severe efficiency bottlenecks on the server. While enabling collaborative training without a central server, existing decentralized FL approaches either focus on the synchronous mechanism that deteriorates FL convergence or ignore device staleness with an asynchronous mechanism, resulting in inferior FL accuracy. In this paper, we propose an Asynchronous Efficient Decentralized FL framework, i.e., AEDFL, in heterogeneous environments with three unique contributions. First, we propose an asynchronous FL system model with an efficient model aggregation method for improving the FL convergence. Second, we propose a dynamic staleness-aware model update approach to achieve superior accuracy. Third, we propose an adaptive sparse training method to reduce communication and computation costs without significant accuracy degradation. Extensive experimentation on four public datasets and four models demonstrates the strength of AEDFL in terms of accuracy (up to 16.3% higher), efficiency (up to 92.9% faster), and computation costs (up to 42.3% lower).

Related papers

• Efficient Federated Learning Using Dynamic Update and Adaptive Pruning with Momentum on Shared Server Data [59.6985168241067]
  Federated Learning (FL) encounters two important problems, i.e., low training efficiency and limited computational resources. We propose a new FL framework, FedDUMAP, to leverage the shared insensitive data on the server and the distributed data in edge devices. Our proposed FL model, FedDUMAP, combines the three original techniques and has a significantly better performance compared with baseline approaches.
  arXiv  Detail & Related papers  (2024-08-11T02:59:11Z)
• Stragglers-Aware Low-Latency Synchronous Federated Learning via Layer-Wise Model Updates [71.81037644563217]
  Synchronous federated learning (FL) is a popular paradigm for collaborative edge learning. As some of the devices may have limited computational resources and varying availability, FL latency is highly sensitive to stragglers. We propose straggler-aware layer-wise federated learning (SALF) that leverages the optimization procedure of NNs via backpropagation to update the global model in a layer-wise fashion.
  arXiv  Detail & Related papers  (2024-03-27T09:14:36Z)
• Momentum Approximation in Asynchronous Private Federated Learning [26.57367597853813]
  momentum approximation can achieve $1.15 textrm--4times$ speed up in convergence compared to existing FLs with momentum. Momentum approximation can be easily integrated in production FL systems with a minor communication and storage cost.
  arXiv  Detail & Related papers  (2024-02-14T15:35:53Z)
• Vertical Federated Learning over Cloud-RAN: Convergence Analysis and System Optimization [82.12796238714589]
  We propose a novel cloud radio access network (Cloud-RAN) based vertical FL system to enable fast and accurate model aggregation. We characterize the convergence behavior of the vertical FL algorithm considering both uplink and downlink transmissions. We establish a system optimization framework by joint transceiver and fronthaul quantization design, for which successive convex approximation and alternate convex search based system optimization algorithms are developed.
  arXiv  Detail & Related papers  (2023-05-04T09:26:03Z)
• FedDUAP: Federated Learning with Dynamic Update and Adaptive Pruning Using Shared Data on the Server [64.94942635929284]
  Federated Learning (FL) suffers from two critical challenges, i.e., limited computational resources and low training efficiency. We propose a novel FL framework, FedDUAP, to exploit the insensitive data on the server and the decentralized data in edge devices. By integrating the two original techniques together, our proposed FL model, FedDUAP, significantly outperforms baseline approaches in terms of accuracy (up to 4.8% higher), efficiency (up to 2.8 times faster), and computational cost (up to 61.9% smaller)
  arXiv  Detail & Related papers  (2022-04-25T10:00:00Z)
• Resource-Efficient and Delay-Aware Federated Learning Design under Edge Heterogeneity [10.702853653891902]
  Federated learning (FL) has emerged as a popular methodology for distributing machine learning across wireless edge devices. In this work, we consider optimizing the tradeoff between model performance and resource utilization in FL. Our proposed StoFedDelAv incorporates a localglobal model combiner into the FL computation step.
  arXiv  Detail & Related papers  (2021-12-27T22:30:15Z)
• Blockchain-enabled Server-less Federated Learning [5.065631761462706]
  We focus on an asynchronous server-less Federated Learning solution empowered by (BC) technology. In contrast to mostly adopted FL approaches, we advocate an asynchronous method whereby model aggregation is done as clients submit their local updates.
  arXiv  Detail & Related papers  (2021-12-15T07:41:23Z)
• HADFL: Heterogeneity-aware Decentralized Federated Learning Framework [1.3780739674182867]
  HADFL is a framework that supports decentralized asynchronous training on heterogeneous devices. It can relieve the central server's communication pressure, efficiently utilize heterogeneous computing power, and can achieve a maximum speedup of 3.15x.
  arXiv  Detail & Related papers  (2021-11-16T07:43:18Z)
• Device Scheduling and Update Aggregation Policies for Asynchronous Federated Learning [72.78668894576515]
  Federated Learning (FL) is a newly emerged decentralized machine learning (ML) framework. We propose an asynchronous FL framework with periodic aggregation to eliminate the straggler issue in FL systems.
  arXiv  Detail & Related papers  (2021-07-23T18:57:08Z)
• Stragglers Are Not Disaster: A Hybrid Federated Learning Algorithm with Delayed Gradients [21.63719641718363]
  Federated learning (FL) is a new machine learning framework which trains a joint model across a large amount of decentralized computing devices. This paper presents a novel FL algorithm, namely Hybrid Federated Learning (HFL), to achieve a learning balance in efficiency and effectiveness.
  arXiv  Detail & Related papers  (2021-02-12T02:27:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.