Revisit the Stability of Vanilla Federated Learning Under Diverse Conditions

• URL: http://arxiv.org/abs/2502.19849v1
• Date: Thu, 27 Feb 2025 07:47:59 GMT
• Title: Revisit the Stability of Vanilla Federated Learning Under Diverse Conditions
• Authors: Youngjoon Lee, Jinu Gong, Sun Choi, Joonhyuk Kang,
• Abstract summary: Federated Learning (FL) is a distributed machine learning paradigm enabling collaborative model training across decentralized clients.<n>We revisit the stability of the vanilla FedAvg algorithm under diverse conditions.
• Score: 3.237380113935023
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated Learning (FL) is a distributed machine learning paradigm enabling collaborative model training across decentralized clients while preserving data privacy. In this paper, we revisit the stability of the vanilla FedAvg algorithm under diverse conditions. Despite its conceptual simplicity, FedAvg exhibits remarkably stable performance compared to more advanced FL techniques. Our experiments assess the performance of various FL methods on blood cell and skin lesion classification tasks using Vision Transformer (ViT). Additionally, we evaluate the impact of different representative classification models and analyze sensitivity to hyperparameter variations. The results consistently demonstrate that, regardless of dataset, classification model employed, or hyperparameter settings, FedAvg maintains robust performance. Given its stability, robust performance without the need for extensive hyperparameter tuning, FedAvg is a safe and efficient choice for FL deployments in resource-constrained hospitals handling medical data. These findings underscore the enduring value of the vanilla FedAvg approach as a trusted baseline for clinical practice.

Related papers

• A Thorough Assessment of the Non-IID Data Impact in Federated Learning [1.89791888476598]
  Decentralized machine learning (FL) allows collaborative machine learning (ML) training among decentralized clients' information, ensuring data privacy. The decentralized nature of FL deals with non-independent and identically distributed (non-IID) data. Despite its importance, experimental studies systematically addressing all types of data heterogeneity (ak.a. non-IIDness) remain scarce. We aim to fill this gap by assessing and quantifying the non-IID effect through a thorough empirical analysis.
  arXiv  Detail & Related papers  (2025-03-21T11:53:36Z)
• Feasible Learning [78.6167929413604]
  We introduce Feasible Learning (FL), a sample-centric learning paradigm where models are trained by solving a feasibility problem that bounds the loss for each training sample.<n>Our empirical analysis, spanning image classification, age regression, and preference optimization in large language models, demonstrates that models trained via FL can learn from data while displaying improved tail behavior compared to ERM, with only a marginal impact on average performance.
  arXiv  Detail & Related papers  (2025-01-24T20:39:38Z)
• Over-the-Air Fair Federated Learning via Multi-Objective Optimization [52.295563400314094]
  We propose an over-the-air fair federated learning algorithm (OTA-FFL) to train fair FL models.<n>Experiments demonstrate the superiority of OTA-FFL in achieving fairness and robust performance.
  arXiv  Detail & Related papers  (2025-01-06T21:16:51Z)
• FedDW: Distilling Weights through Consistency Optimization in Heterogeneous Federated Learning [14.477559543490242]
  Federated Learning (FL) is an innovative distributed machine learning paradigm that enables neural network training across devices without centralizing data.<n>Previous research shows that in IID environments, the parameter structure of the model is expected to adhere to certain specific consistency principles.<n>This paper identifies the consistency between the two and leverages it to regulate training, underpinning our proposed FedDW framework.<n> Experimental results show FedDW outperforms 10 state-of-the-art FL methods, improving accuracy by an average of 3% in highly heterogeneous settings.
  arXiv  Detail & Related papers  (2024-12-05T12:32:40Z)
• FedGS: Federated Gradient Scaling for Heterogeneous Medical Image Segmentation [0.4499833362998489]
  We propose FedGS, a novel FL aggregation method, to improve segmentation performance on small, under-represented targets. FedGS demonstrates superior performance over FedAvg, particularly for small lesions, across PolypGen and LiTS datasets.
  arXiv  Detail & Related papers  (2024-08-21T15:26:21Z)
• On ADMM in Heterogeneous Federated Learning: Personalization, Robustness, and Fairness [16.595935469099306]
  We propose FLAME, an optimization framework by utilizing the alternating direction method of multipliers (ADMM) to train personalized and global models. Our theoretical analysis establishes the global convergence and two kinds of convergence rates for FLAME under mild assumptions. Our experimental findings show that FLAME outperforms state-of-the-art methods in convergence and accuracy, and it achieves higher test accuracy under various attacks.
  arXiv  Detail & Related papers  (2024-07-23T11:35:42Z)
• FLASH: Federated Learning Across Simultaneous Heterogeneities [54.80435317208111]
  FLASH(Federated Learning Across Simultaneous Heterogeneities) is a lightweight and flexible client selection algorithm. It outperforms state-of-the-art FL frameworks under extensive sources of Heterogeneities. It achieves substantial and consistent improvements over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-02-13T20:04:39Z)
• Federated Conformal Predictors for Distributed Uncertainty Quantification [83.50609351513886]
  Conformal prediction is emerging as a popular paradigm for providing rigorous uncertainty quantification in machine learning. In this paper, we extend conformal prediction to the federated learning setting. We propose a weaker notion of partial exchangeability, better suited to the FL setting, and use it to develop the Federated Conformal Prediction framework.
  arXiv  Detail & Related papers  (2023-05-27T19:57:27Z)
• Reliable Federated Disentangling Network for Non-IID Domain Feature [62.73267904147804]
  In this paper, we propose a novel reliable federated disentangling network, termed RFedDis. To the best of our knowledge, our proposed RFedDis is the first work to develop an FL approach based on evidential uncertainty combined with feature disentangling. Our proposed RFedDis provides outstanding performance with a high degree of reliability as compared to other state-of-the-art FL approaches.
  arXiv  Detail & Related papers  (2023-01-30T11:46:34Z)
• FedPseudo: Pseudo value-based Deep Learning Models for Federated Survival Analysis [9.659041001051415]
  We propose a first-of-its-kind, pseudo value-based deep learning model for federated survival analysis called FedPseudo. Our proposed FL framework achieves similar performance as the best centrally trained deep survival analysis model.
  arXiv  Detail & Related papers  (2022-07-12T01:10:36Z)
• Label-Efficient Self-Supervised Federated Learning for Tackling Data Heterogeneity in Medical Imaging [23.08596805950814]
  We present a robust and label-efficient self-supervised FL framework for medical image analysis. Specifically, we introduce a novel distributed self-supervised pre-training paradigm into the existing FL pipeline. We show that our self-supervised FL algorithm generalizes well to out-of-distribution data and learns federated models more effectively in limited label scenarios.
  arXiv  Detail & Related papers  (2022-05-17T18:33:43Z)
• MS Lesion Segmentation: Revisiting Weighting Mechanisms for Federated Learning [92.91544082745196]
  Federated learning (FL) has been widely employed for medical image analysis. FL's performance is limited for multiple sclerosis (MS) lesion segmentation tasks. We propose the first FL MS lesion segmentation framework via two effective re-weighting mechanisms.
  arXiv  Detail & Related papers  (2022-05-03T14:06:03Z)
• Do Gradient Inversion Attacks Make Federated Learning Unsafe? [70.0231254112197]
  Federated learning (FL) allows the collaborative training of AI models without needing to share raw data. Recent works on the inversion of deep neural networks from model gradients raised concerns about the security of FL in preventing the leakage of training data. In this work, we show that these attacks presented in the literature are impractical in real FL use-cases and provide a new baseline attack.
  arXiv  Detail & Related papers  (2022-02-14T18:33:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.