An Empirical Study of the Impact of Federated Learning on Machine Learning Model Accuracy

• URL: http://arxiv.org/abs/2503.20768v2
• Date: Thu, 27 Mar 2025 02:16:06 GMT
• Title: An Empirical Study of the Impact of Federated Learning on Machine Learning Model Accuracy
• Authors: Haotian Yang, Zhuoran Wang, Benson Chou, Sophie Xu, Hao Wang, Jingxian Wang, Qizhen Zhang,
• Abstract summary: Federated Learning (FL) enables distributed ML model training on private user data at the global scale.<n>We investigate, systematically, how this learning paradigm can affect the accuracy of state-of-the-art ML models for a variety of ML tasks.
• Score: 6.907897341928317
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Federated Learning (FL) enables distributed ML model training on private user data at the global scale. Despite the potential of FL demonstrated in many domains, an in-depth view of its impact on model accuracy remains unclear. In this paper, we investigate, systematically, how this learning paradigm can affect the accuracy of state-of-the-art ML models for a variety of ML tasks. We present an empirical study that involves various data types: text, image, audio, and video, and FL configuration knobs: data distribution, FL scale, client sampling, and local and global computations. Our experiments are conducted in a unified FL framework to achieve high fidelity, with substantial human efforts and resource investments. Based on the results, we perform a quantitative analysis of the impact of FL, and highlight challenging scenarios where applying FL degrades the accuracy of the model drastically and identify cases where the impact is negligible. The detailed and extensive findings can benefit practical deployments and future development of FL.

Related papers

• Feasible Learning [78.6167929413604]
  We introduce Feasible Learning (FL), a sample-centric learning paradigm where models are trained by solving a feasibility problem that bounds the loss for each training sample.<n>Our empirical analysis, spanning image classification, age regression, and preference optimization in large language models, demonstrates that models trained via FL can learn from data while displaying improved tail behavior compared to ERM, with only a marginal impact on average performance.
  arXiv  Detail & Related papers  (2025-01-24T20:39:38Z)
• Unlocking the Potential of Model Calibration in Federated Learning [15.93119575317457]
  We propose Non-Uniform for Federated Learning (NUCFL), a generic framework that integrates FL with the concept of model calibration.<n>OurFL addresses this challenge by dynamically adjusting the model calibration based on relationships between each clients local model and the global model in FL.<n>By doing so,FL effectively aligns calibration needs for the global model while not sacrificing accuracy.
  arXiv  Detail & Related papers  (2024-09-07T20:11:11Z)
• Can We Theoretically Quantify the Impacts of Local Updates on the Generalization Performance of Federated Learning? [50.03434441234569]
  Federated Learning (FL) has gained significant popularity due to its effectiveness in training machine learning models across diverse sites without requiring direct data sharing. While various algorithms have shown that FL with local updates is a communication-efficient distributed learning framework, the generalization performance of FL with local updates has received comparatively less attention.
  arXiv  Detail & Related papers  (2024-09-05T19:00:18Z)
• A Survey on Efficient Federated Learning Methods for Foundation Model Training [62.473245910234304]
  Federated Learning (FL) has become an established technique to facilitate privacy-preserving collaborative training across a multitude of clients. In the wake of Foundation Models (FM), the reality is different for many deep learning applications. We discuss the benefits and drawbacks of parameter-efficient fine-tuning (PEFT) for FL applications.
  arXiv  Detail & Related papers  (2024-01-09T10:22:23Z)
• FS-Real: Towards Real-World Cross-Device Federated Learning [60.91678132132229]
  Federated Learning (FL) aims to train high-quality models in collaboration with distributed clients while not uploading their local data. There is still a considerable gap between the flourishing FL research and real-world scenarios, mainly caused by the characteristics of heterogeneous devices and its scales. We propose an efficient and scalable prototyping system for real-world cross-device FL, FS-Real.
  arXiv  Detail & Related papers  (2023-03-23T15:37:17Z)
• Automated Federated Learning in Mobile Edge Networks -- Fast Adaptation and Convergence [83.58839320635956]
  Federated Learning (FL) can be used in mobile edge networks to train machine learning models in a distributed manner. Recent FL has been interpreted within a Model-Agnostic Meta-Learning (MAML) framework, which brings FL significant advantages in fast adaptation and convergence over heterogeneous datasets. This paper addresses how much benefit MAML brings to FL and how to maximize such benefit over mobile edge networks.
  arXiv  Detail & Related papers  (2023-03-23T02:42:10Z)
• When Federated Learning Meets Pre-trained Language Models' Parameter-Efficient Tuning Methods [22.16636947999123]
  We introduce various parameter-efficient tuning (PETuning) methods into federated learning. Specifically, we provide a holistic empirical study of representative PLMs tuning methods in FL. Overall communication overhead can be significantly reduced by locally tuning and globally aggregating lightweight model parameters.
  arXiv  Detail & Related papers  (2022-12-20T06:44:32Z)
• Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning [86.59588262014456]
  Federated Learning (FL) is an emerging distributed learning paradigm under privacy constraint. We propose a data-free knowledge distillation method to fine-tune the global model in the server (FedFTG) Our FedFTG significantly outperforms the state-of-the-art (SOTA) FL algorithms and can serve as a strong plugin for enhancing FedAvg, FedProx, FedDyn, and SCAFFOLD.
  arXiv  Detail & Related papers  (2022-03-17T11:18:17Z)
• Critical Learning Periods in Federated Learning [11.138980572551066]
  Federated learning (FL) is a popular technique to train machine learning (ML) models with decentralized data. We show that the final test accuracy of FL is dramatically affected by the early phase of the training process.
  arXiv  Detail & Related papers  (2021-09-12T21:06:07Z)
• FedScale: Benchmarking Model and System Performance of Federated Learning [4.1617240682257925]
  FedScale is a set of challenging and realistic benchmark datasets for federated learning (FL) research. FedScale is open-source with permissive licenses and actively maintained.
  arXiv  Detail & Related papers  (2021-05-24T15:55:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.