Kuramoto-FedAvg: Using Synchronization Dynamics to Improve Federated Learning Optimization under Statistical Heterogeneity

• URL: http://arxiv.org/abs/2505.19605v1
• Date: Mon, 26 May 2025 07:16:00 GMT
• Title: Kuramoto-FedAvg: Using Synchronization Dynamics to Improve Federated Learning Optimization under Statistical Heterogeneity
• Authors: Aggrey Muhebwa, Khotso Selialia, Fatima Anwar, Khalid K. Osman,
• Abstract summary: Federated learning on heterogeneous (non-IID) client data experiences slow convergence due to client drift.<n>We propose Kuramoto-FedAvg, a federated optimization algorithm that reframes the weight aggregation step as a synchronization problem.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated learning on heterogeneous (non-IID) client data experiences slow convergence due to client drift. To address this challenge, we propose Kuramoto-FedAvg, a federated optimization algorithm that reframes the weight aggregation step as a synchronization problem inspired by the Kuramoto model of coupled oscillators. The server dynamically weighs each client's update based on its phase alignment with the global update, amplifying contributions that align with the global gradient direction while minimizing the impact of updates that are out of phase. We theoretically prove that this synchronization mechanism reduces client drift, providing a tighter convergence bound compared to the standard FedAvg under heterogeneous data distributions. Empirical validation supports our theoretical findings, showing that Kuramoto-FedAvg significantly accelerates convergence and improves accuracy across multiple benchmark datasets. Our work highlights the potential of coordination and synchronization-based strategies for managing gradient diversity and accelerating federated optimization in realistic non-IID settings.

Related papers

• Asynchronous Federated Learning with non-convex client objective functions and heterogeneous dataset [0.9208007322096533]
  Tosampling Federated Learning (FL) enables collaborative model across decentralized devices while preserving stale data privacy.<n>Asynchronous Learning (AFL) addresses these by allowing clients to update independently, improving scalability and reducing delays synchronization.<n>Our framework accommodates variations in data power, distribution, and communication, making it practical for real world applications.
  arXiv  Detail & Related papers  (2025-08-03T09:06:42Z)
• Adaptive Deadline and Batch Layered Synchronized Federated Learning [66.93447103966439]
  Federated learning (FL) enables collaborative model training across distributed edge devices while preserving data privacy, and typically operates in a round-based synchronous manner.<n>We propose ADEL-FL, a novel framework that jointly optimize per-round deadlines and user-specific batch sizes for layer-wise aggregation.
  arXiv  Detail & Related papers  (2025-05-29T19:59:18Z)
• Asynchronous Federated Learning: A Scalable Approach for Decentralized Machine Learning [0.9208007322096533]
  Federated Learning (FL) has emerged as a powerful paradigm for decentralized machine learning, enabling collaborative model training across diverse clients without sharing raw data.<n>Traditional FL approaches often face limitations in scalability and efficiency due to their reliance on synchronous client updates.<n>We propose an Asynchronous Federated Learning (AFL) algorithm, which allows clients to update the global model independently and asynchronously.
  arXiv  Detail & Related papers  (2024-12-23T17:11:02Z)
• Boosting the Performance of Decentralized Federated Learning via Catalyst Acceleration [66.43954501171292]
  We introduce Catalyst Acceleration and propose an acceleration Decentralized Federated Learning algorithm called DFedCata. DFedCata consists of two main components: the Moreau envelope function, which addresses parameter inconsistencies, and Nesterov's extrapolation step, which accelerates the aggregation phase. Empirically, we demonstrate the advantages of the proposed algorithm in both convergence speed and generalization performance on CIFAR10/100 with various non-iid data distributions.
  arXiv  Detail & Related papers  (2024-10-09T06:17:16Z)
• Aiding Global Convergence in Federated Learning via Local Perturbation and Mutual Similarity Information [6.767885381740953]
  Federated learning has emerged as a distributed optimization paradigm. We propose a novel modified framework wherein each client locally performs a perturbed gradient step. We show that our algorithm speeds convergence up to a margin of 30 global rounds compared with FedAvg.
  arXiv  Detail & Related papers  (2024-10-07T23:14:05Z)
• FADAS: Towards Federated Adaptive Asynchronous Optimization [56.09666452175333]
  Federated learning (FL) has emerged as a widely adopted training paradigm for privacy-preserving machine learning. This paper introduces federated adaptive asynchronous optimization, named FADAS, a novel method that incorporates asynchronous updates into adaptive federated optimization with provable guarantees. We rigorously establish the convergence rate of the proposed algorithms and empirical results demonstrate the superior performance of FADAS over other asynchronous FL baselines.
  arXiv  Detail & Related papers  (2024-07-25T20:02:57Z)
• Stragglers-Aware Low-Latency Synchronous Federated Learning via Layer-Wise Model Updates [71.81037644563217]
  Synchronous federated learning (FL) is a popular paradigm for collaborative edge learning. As some of the devices may have limited computational resources and varying availability, FL latency is highly sensitive to stragglers. We propose straggler-aware layer-wise federated learning (SALF) that leverages the optimization procedure of NNs via backpropagation to update the global model in a layer-wise fashion.
  arXiv  Detail & Related papers  (2024-03-27T09:14:36Z)
• Federated Learning based on Pruning and Recovery [0.0]
  This framework integrates asynchronous learning algorithms and pruning techniques. It addresses the inefficiencies of traditional federated learning algorithms in scenarios involving heterogeneous devices. It also tackles the staleness issue and inadequate training of certain clients in asynchronous algorithms.
  arXiv  Detail & Related papers  (2024-03-16T14:35:03Z)
• Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscape [59.841889495864386]
  In federated learning (FL), a cluster of local clients are chaired under the coordination of a global server. Clients are prone to overfit into their own optima, which extremely deviates from the global objective. ttfamily FedSMOO adopts a dynamic regularizer to guarantee the local optima towards the global objective. Our theoretical analysis indicates that ttfamily FedSMOO achieves fast $mathcalO (1/T)$ convergence rate with low bound generalization.
  arXiv  Detail & Related papers  (2023-05-19T10:47:44Z)
• FedSkip: Combatting Statistical Heterogeneity with Federated Skip Aggregation [95.85026305874824]
  We introduce a data-driven approach called FedSkip to improve the client optima by periodically skipping federated averaging and scattering local models to the cross devices. We conduct extensive experiments on a range of datasets to demonstrate that FedSkip achieves much higher accuracy, better aggregation efficiency and competing communication efficiency.
  arXiv  Detail & Related papers  (2022-12-14T13:57:01Z)
• A General Theory for Federated Optimization with Asynchronous and Heterogeneous Clients Updates [10.609815608017065]
  We extend the standard FedAvg aggregation scheme by introducing aggregation weights to represent the variability of the clients update time. Our formalism applies to the general federated setting where clients have heterogeneous datasets and perform at least one step of gradient descent. We develop in this work FedFix, a novel extension of FedAvg enabling efficient asynchronous federated training while preserving the convergence stability of synchronous aggregation.
  arXiv  Detail & Related papers  (2022-06-21T08:46:05Z)
• From Deterioration to Acceleration: A Calibration Approach to Rehabilitating Step Asynchronism in Federated Optimization [13.755421424240048]
  We propose a new algorithm textttFedaGrac, which calibrates the local direction to a predictive global orientation. We theoretically prove that textttFedaGrac holds an improved order of convergence rate than the state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-12-17T07:26:31Z)
• Faster Non-Convex Federated Learning via Global and Local Momentum [57.52663209739171]
  textttFedGLOMO is the first (first-order) FLtexttFedGLOMO algorithm. Our algorithm is provably optimal even with communication between the clients and the server.
  arXiv  Detail & Related papers  (2020-12-07T21:05:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.