Corrected with the Latest Version: Make Robust Asynchronous Federated Learning Possible

• URL: http://arxiv.org/abs/2504.04081v2
• Date: Wed, 09 Apr 2025 05:42:03 GMT
• Title: Corrected with the Latest Version: Make Robust Asynchronous Federated Learning Possible
• Authors: Chaoyi Lu, Yiding Sun, Pengbo Li, Zhichuan Yang,
• Abstract summary: This paper proposes an asynchronous federated learning version correction algorithm based on knowledge distillation, named FedADT.<n>FedADT applies knowledge distillation before aggregating gradients, using the latest global model to correct outdated information, thus effectively reducing the negative impact of outdated gradients on the training process.<n>We conducted experimental comparisons with several classical algorithms, and the results demonstrate that FedADT achieves significant improvements over other asynchronous methods and outperforms all methods in terms of convergence speed.
• Score: 2.663489028501814
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As an emerging paradigm of federated learning, asynchronous federated learning offers significant speed advantages over traditional synchronous federated learning. Unlike synchronous federated learning, which requires waiting for all clients to complete updates before aggregation, asynchronous federated learning aggregates the models that have arrived in realtime, greatly improving training speed. However, this mechanism also introduces the issue of client model version inconsistency. When the differences between models of different versions during aggregation become too large, it may lead to conflicts, thereby reducing the models accuracy. To address this issue, this paper proposes an asynchronous federated learning version correction algorithm based on knowledge distillation, named FedADT. FedADT applies knowledge distillation before aggregating gradients, using the latest global model to correct outdated information, thus effectively reducing the negative impact of outdated gradients on the training process. Additionally, FedADT introduces an adaptive weighting function that adjusts the knowledge distillation weight according to different stages of training, helps mitigate the misleading effects caused by the poorer performance of the global model in the early stages of training. This method significantly improves the overall performance of asynchronous federated learning without adding excessive computational overhead. We conducted experimental comparisons with several classical algorithms, and the results demonstrate that FedADT achieves significant improvements over other asynchronous methods and outperforms all methods in terms of convergence speed.

Related papers

• SEAFL: Enhancing Efficiency in Semi-Asynchronous Federated Learning through Adaptive Aggregation and Selective Training [26.478852701376294]
  We present em SEAFL, a novel FL framework designed to mitigate both the straggler and the stale model challenges in semi-asynchronous FL.<n>em SEAFL dynamically assigns weights to uploaded models during aggregation based on their staleness and importance to the current global model.<n>We evaluate the effectiveness of em SEAFL through extensive experiments on three benchmark datasets.
  arXiv  Detail & Related papers  (2025-02-22T05:13:53Z)
• FedFa: A Fully Asynchronous Training Paradigm for Federated Learning [14.4313600357833]
  Federated learning is an efficient decentralized training paradigm for scaling the machine learning model training on a large number of devices. Recent state-of-the-art solutions propose using semi-asynchronous approaches to mitigate the waiting time cost with guaranteed convergence. We propose a full asynchronous training paradigm, called FedFa, which can guarantee model convergence and eliminate the waiting time completely.
  arXiv  Detail & Related papers  (2024-04-17T02:46:59Z)
• Stragglers-Aware Low-Latency Synchronous Federated Learning via Layer-Wise Model Updates [71.81037644563217]
  Synchronous federated learning (FL) is a popular paradigm for collaborative edge learning. As some of the devices may have limited computational resources and varying availability, FL latency is highly sensitive to stragglers. We propose straggler-aware layer-wise federated learning (SALF) that leverages the optimization procedure of NNs via backpropagation to update the global model in a layer-wise fashion.
  arXiv  Detail & Related papers  (2024-03-27T09:14:36Z)
• Federated Learning based on Pruning and Recovery [0.0]
  This framework integrates asynchronous learning algorithms and pruning techniques. It addresses the inefficiencies of traditional federated learning algorithms in scenarios involving heterogeneous devices. It also tackles the staleness issue and inadequate training of certain clients in asynchronous algorithms.
  arXiv  Detail & Related papers  (2024-03-16T14:35:03Z)
• Knowledge Rumination for Client Utility Evaluation in Heterogeneous Federated Learning [12.50871784200551]
  Federated Learning (FL) allows several clients to cooperatively train machine learning models without disclosing the raw data.<n>Non-IID data and stale models pose significant challenges to AFL, as they can diminish the practicality of the global model and even lead to training failures.<n>We propose a novel AFL framework called Federated Historical Learning (FedHist), which effectively addresses the challenges posed by both Non-IID data and gradient staleness.
  arXiv  Detail & Related papers  (2023-12-16T11:40:49Z)
• FedLALR: Client-Specific Adaptive Learning Rates Achieve Linear Speedup for Non-IID Data [54.81695390763957]
  Federated learning is an emerging distributed machine learning method. We propose a heterogeneous local variant of AMSGrad, named FedLALR, in which each client adjusts its learning rate. We show that our client-specified auto-tuned learning rate scheduling can converge and achieve linear speedup with respect to the number of clients.
  arXiv  Detail & Related papers  (2023-09-18T12:35:05Z)
• Faster Adaptive Federated Learning [84.38913517122619]
  Federated learning has attracted increasing attention with the emergence of distributed data. In this paper, we propose an efficient adaptive algorithm (i.e., FAFED) based on momentum-based variance reduced technique in cross-silo FL.
  arXiv  Detail & Related papers  (2022-12-02T05:07:50Z)
• AsyncFedED: Asynchronous Federated Learning with Euclidean Distance based Adaptive Weight Aggregation [17.57059932879715]
  In an asynchronous learning framework, a server updates the global model once it receives an update from a client instead of waiting for all the updates to arrive as in the setting. A proposed adaptive weight aggregation algorithm, referred to as AsyncFedED, is presented.
  arXiv  Detail & Related papers  (2022-05-27T07:18:11Z)
• Revisiting Consistency Regularization for Semi-Supervised Learning [80.28461584135967]
  We propose an improved consistency regularization framework by a simple yet effective technique, FeatDistLoss. Experimental results show that our model defines a new state of the art for various datasets and settings.
  arXiv  Detail & Related papers  (2021-12-10T20:46:13Z)
• Communication-Efficient Federated Learning with Compensated Overlap-FedAvg [22.636184975591004]
  Federated learning is proposed to perform model training by multiple clients' combined data without the dataset sharing within the cluster. We propose Overlap-FedAvg, a framework that parallels the model training phase with model uploading & downloading phase. Overlap-FedAvg is further developed with a hierarchical computing strategy, a data compensation mechanism and a nesterov accelerated gradients(NAG) algorithm.
  arXiv  Detail & Related papers  (2020-12-12T02:50:09Z)
• Fast-Convergent Federated Learning [82.32029953209542]
  Federated learning is a promising solution for distributing machine learning tasks through modern networks of mobile devices. We propose a fast-convergent federated learning algorithm, called FOLB, which performs intelligent sampling of devices in each round of model training.
  arXiv  Detail & Related papers  (2020-07-26T14:37:51Z)
• Federated Residual Learning [53.77128418049985]
  We study a new form of federated learning where the clients train personalized local models and make predictions jointly with the server-side shared model. Using this new federated learning framework, the complexity of the central shared model can be minimized while still gaining all the performance benefits that joint training provides.
  arXiv  Detail & Related papers  (2020-03-28T19:55:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.