Reviving Stale Updates: Data-Free Knowledge Distillation for Asynchronous Federated Learning

• URL: http://arxiv.org/abs/2511.00655v1
• Date: Sat, 01 Nov 2025 18:35:20 GMT
• Title: Reviving Stale Updates: Data-Free Knowledge Distillation for Asynchronous Federated Learning
• Authors: Baris Askin, Holger R. Roth, Zhenyu Sun, Carlee Joe-Wong, Gauri Joshi, Ziyue Xu,
• Abstract summary: Federated Learning (FL) enables collaborative model training across distributed clients without sharing raw data.<n>Asynchronous Federated Learning (AFL) alleviates this issue by allowing clients to communicate independently.<n>We propose FedRevive, an asynchronous FL framework that revives stale updates through data-free knowledge distillation (DFKD)
• Score: 28.983471880773305
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated Learning (FL) enables collaborative model training across distributed clients without sharing raw data, yet its scalability is limited by synchronization overhead. Asynchronous Federated Learning (AFL) alleviates this issue by allowing clients to communicate independently, thereby improving wall-clock efficiency in large-scale, heterogeneous environments. However, this asynchrony introduces stale updates (client updates computed on outdated global models) that can destabilize optimization and hinder convergence. We propose FedRevive, an asynchronous FL framework that revives stale updates through data-free knowledge distillation (DFKD). FedRevive integrates parameter-space aggregation with a lightweight, server-side DFKD process that transfers knowledge from stale client models to the current global model without access to real or public data. A meta-learned generator synthesizes pseudo-samples, which enables multi-teacher distillation. A hybrid aggregation scheme that combines raw updates with DFKD updates effectively mitigates staleness while retaining the scalability of AFL. Experiments on various vision and text benchmarks show that FedRevive achieves faster training up to 32.1% and higher final accuracy up to 21.5% compared to asynchronous baselines.

Related papers

• Stragglers Can Contribute More: Uncertainty-Aware Distillation for Asynchronous Federated Learning [61.249748418757946]
  Asynchronous federated learning (FL) has recently gained attention for its enhanced efficiency and scalability.<n>We propose FedEcho, a novel framework that incorporates uncertainty-aware distillation to enhance the asynchronous FL performances.<n>We demonstrate that FedEcho consistently outperforms existing asynchronous federated learning baselines.
  arXiv  Detail & Related papers  (2025-11-25T06:25:25Z)
• CO-PFL: Contribution-Oriented Personalized Federated Learning for Heterogeneous Networks [51.43780477302533]
  Contribution-Oriented PFL (CO-PFL) is a novel algorithm that dynamically estimates each client's contribution for global aggregation.<n>CO-PFL consistently surpasses state-of-the-art methods in robustness in personalization accuracy, robustness, scalability and convergence stability.
  arXiv  Detail & Related papers  (2025-10-23T05:10:06Z)
• Asynchronous Federated Learning with non-convex client objective functions and heterogeneous dataset [0.9208007322096533]
  Tosampling Federated Learning (FL) enables collaborative model across decentralized devices while preserving stale data privacy.<n>Asynchronous Learning (AFL) addresses these by allowing clients to update independently, improving scalability and reducing delays synchronization.<n>Our framework accommodates variations in data power, distribution, and communication, making it practical for real world applications.
  arXiv  Detail & Related papers  (2025-08-03T09:06:42Z)
• Adaptive Deadline and Batch Layered Synchronized Federated Learning [66.93447103966439]
  Federated learning (FL) enables collaborative model training across distributed edge devices while preserving data privacy, and typically operates in a round-based synchronous manner.<n>We propose ADEL-FL, a novel framework that jointly optimize per-round deadlines and user-specific batch sizes for layer-wise aggregation.
  arXiv  Detail & Related papers  (2025-05-29T19:59:18Z)
• FRAIN to Train: A Fast-and-Reliable Solution for Decentralized Federated Learning [1.1510009152620668]
  asynchronous learning (FL) enables collaborative model training across distributed clients while preserving data locality.<n>We introduce Fast-and-Reliable AI Network, FRAIN, a new FL method that mitigates these limitations by incorporating two key ideas.<n>Experiments with a CNN image-classification model and a Transformer-based language model demonstrate that FRAIN achieves more stable and robust convergence than FedAvg, FedAsync, and BRAIN.
  arXiv  Detail & Related papers  (2025-05-07T08:20:23Z)
• FedAST: Federated Asynchronous Simultaneous Training [27.492821176616815]
  Federated Learning (FL) enables devices or clients to collaboratively train machine learning (ML) models without sharing their private data. Much of the existing work in FL focuses on efficiently learning a model for a single task. In this paper, we propose simultaneous training of multiple FL models using a common set of datasets.
  arXiv  Detail & Related papers  (2024-06-01T05:14:20Z)
• An Aggregation-Free Federated Learning for Tackling Data Heterogeneity [50.44021981013037]
  Federated Learning (FL) relies on the effectiveness of utilizing knowledge from distributed datasets. Traditional FL methods adopt an aggregate-then-adapt framework, where clients update local models based on a global model aggregated by the server from the previous training round. We introduce FedAF, a novel aggregation-free FL algorithm.
  arXiv  Detail & Related papers  (2024-04-29T05:55:23Z)
• Stragglers-Aware Low-Latency Synchronous Federated Learning via Layer-Wise Model Updates [71.81037644563217]
  Synchronous federated learning (FL) is a popular paradigm for collaborative edge learning. As some of the devices may have limited computational resources and varying availability, FL latency is highly sensitive to stragglers. We propose straggler-aware layer-wise federated learning (SALF) that leverages the optimization procedure of NNs via backpropagation to update the global model in a layer-wise fashion.
  arXiv  Detail & Related papers  (2024-03-27T09:14:36Z)
• Enhancing Convergence in Federated Learning: A Contribution-Aware Asynchronous Approach [0.0]
  Federated Learning (FL) is a distributed machine learning paradigm that allows clients to train models on their data while preserving their privacy. FL algorithms, such as Federated Averaging (FedAvg) and its variants, have been shown to converge well in many scenarios. However, these methods require clients to upload their local updates to the server in a synchronous manner, which can be slow and unreliable in realistic FL settings. We propose a contribution-aware asynchronous FL method that takes into account the staleness and statistical heterogeneity of the received updates.
  arXiv  Detail & Related papers  (2024-02-16T12:10:53Z)
• AEDFL: Efficient Asynchronous Decentralized Federated Learning with Heterogeneous Devices [61.66943750584406]
  We propose an Asynchronous Efficient Decentralized FL framework, i.e., AEDFL, in heterogeneous environments. First, we propose an asynchronous FL system model with an efficient model aggregation method for improving the FL convergence. Second, we propose a dynamic staleness-aware model update approach to achieve superior accuracy. Third, we propose an adaptive sparse training method to reduce communication and computation costs without significant accuracy degradation.
  arXiv  Detail & Related papers  (2023-12-18T05:18:17Z)
• TimelyFL: Heterogeneity-aware Asynchronous Federated Learning with Adaptive Partial Training [17.84692242938424]
  TimelyFL is a heterogeneous-aware asynchronous Federated Learning framework with adaptive partial training. We show that TimelyFL improves participation rate by 21.13%, harvests 1.28x - 2.89x more efficiency on convergence rate, and provides a 6.25% increment on test accuracy.
  arXiv  Detail & Related papers  (2023-04-14T06:26:08Z)
• Over-the-Air Federated Learning from Heterogeneous Data [107.05618009955094]
  Federated learning (FL) is a framework for distributed learning of centralized models. We develop a Convergent OTA FL (COTAF) algorithm which enhances the common local gradient descent (SGD) FL algorithm. We numerically show that the precoding induced by COTAF notably improves the convergence rate and the accuracy of models trained via OTA FL.
  arXiv  Detail & Related papers  (2020-09-27T08:28:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.