Convergence Analysis of Sequential Federated Learning on Heterogeneous Data

• URL: http://arxiv.org/abs/2311.03154v2
• Date: Wed, 8 May 2024 07:08:34 GMT
• Title: Convergence Analysis of Sequential Federated Learning on Heterogeneous Data
• Authors: Yipeng Li, Xinchen Lyu,
• Abstract summary: There are two categories of methods in Federated Learning (FL) for joint training across multiple clients: i) parallel FL (PFL), where clients train models in a parallel manner; and ii) FL (SFL) where clients train in a sequential manner. In this paper, we establish the convergence guarantees SFL on heterogeneous data is still lacking. Experimental results validate the counterintuitive analysis result that SFL outperforms PFL on extremely heterogeneous data in cross-device settings.
• Score: 5.872735527071425
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: There are two categories of methods in Federated Learning (FL) for joint training across multiple clients: i) parallel FL (PFL), where clients train models in a parallel manner; and ii) sequential FL (SFL), where clients train models in a sequential manner. In contrast to that of PFL, the convergence theory of SFL on heterogeneous data is still lacking. In this paper, we establish the convergence guarantees of SFL for strongly/general/non-convex objectives on heterogeneous data. The convergence guarantees of SFL are better than that of PFL on heterogeneous data with both full and partial client participation. Experimental results validate the counterintuitive analysis result that SFL outperforms PFL on extremely heterogeneous data in cross-device settings.

Related papers

• FuseFL: One-Shot Federated Learning through the Lens of Causality with Progressive Model Fusion [48.90879664138855]
  One-shot Federated Learning (OFL) significantly reduces communication costs in FL by aggregating trained models only once. However, the performance of advanced OFL methods is far behind the normal FL. We propose a novel learning approach to endow OFL with superb performance and low communication and storage costs, termed as FuseFL.
  arXiv  Detail & Related papers  (2024-10-27T09:07:10Z)
• Sharp Bounds for Sequential Federated Learning on Heterogeneous Data [5.872735527071425]
  There are two paradigms in Learning (FL): parallel FL (PFL) and sequential FL (SFL) In contrast to that of PFL, convergence theory SFL data is still lacking. We derive the upper bounds for strongly convex, general convex and sequential non-counterintuitive objective functions. We compare the upper bounds SFL with those on heterogeneous PFL data.
  arXiv  Detail & Related papers  (2024-05-02T09:58:49Z)
• Convergence Analysis of Split Federated Learning on Heterogeneous Data [10.61370409320618]
  Split learning (SFL) is a recent distributed approach for collaborative model training among multiple clients. In SFL, a global model is typically split into two parts, where clients train one part in a parallel federated manner, and the other trains the other. We provide convergence analysis of SFL for strongly convex and general objectives on heterogeneous data.
  arXiv  Detail & Related papers  (2024-02-23T07:59:23Z)
• MergeSFL: Split Federated Learning with Feature Merging and Batch Size Regulation [27.159538773609917]
  Federated learning (FL) is a technique for edge AI to mine valuable knowledge in edge computing (EC) systems. We propose a novel SFL framework, termed MergeSFL, by incorporating feature merging and batch size regulation in SFL. We show that MergeSFL can improve the final model accuracy by 5.82% to 26.22%, with a speedup by about 1.74x to 4.14x, compared to the baselines.
  arXiv  Detail & Related papers  (2023-11-22T12:25:02Z)
• PFL-GAN: When Client Heterogeneity Meets Generative Models in Personalized Federated Learning [55.930403371398114]
  We propose a novel generative adversarial network (GAN) sharing and aggregation strategy for personalized learning (PFL) PFL-GAN addresses the client heterogeneity in different scenarios. More specially, we first learn the similarity among clients and then develop an weighted collaborative data aggregation. The empirical results through the rigorous experimentation on several well-known datasets demonstrate the effectiveness of PFL-GAN.
  arXiv  Detail & Related papers  (2023-08-23T22:38:35Z)
• Hierarchical Personalized Federated Learning Over Massive Mobile Edge Computing Networks [95.39148209543175]
  We propose hierarchical PFL (HPFL), an algorithm for deploying PFL over massive MEC networks. HPFL combines the objectives of training loss minimization and round latency minimization while jointly determining the optimal bandwidth allocation.
  arXiv  Detail & Related papers  (2023-03-19T06:00:05Z)
• Semi-Synchronous Personalized Federated Learning over Mobile Edge Networks [88.50555581186799]
  We propose a semi-synchronous PFL algorithm, termed as Semi-Synchronous Personalized FederatedAveraging (PerFedS$2$), over mobile edge networks. We derive an upper bound of the convergence rate of PerFedS2 in terms of the number of participants per global round and the number of rounds. Experimental results verify the effectiveness of PerFedS2 in saving training time as well as guaranteeing the convergence of training loss.
  arXiv  Detail & Related papers  (2022-09-27T02:12:43Z)
• pFL-Bench: A Comprehensive Benchmark for Personalized Federated Learning [42.819532536636835]
  We propose the first comprehensive pFL benchmark, pFL-Bench, for rapid, reproducible, standardized and thorough pFL evaluation. The proposed benchmark contains more than 10 datasets in diverse application domains with unified data partition and realistic heterogeneous settings. We highlight the benefits and potential of state-of-the-art pFL methods and hope pFL-Bench enables further pFL research and broad applications.
  arXiv  Detail & Related papers  (2022-06-08T02:51:59Z)
• Achieving Personalized Federated Learning with Sparse Local Models [75.76854544460981]
  Federated learning (FL) is vulnerable to heterogeneously distributed data. To counter this issue, personalized FL (PFL) was proposed to produce dedicated local models for each individual user. Existing PFL solutions either demonstrate unsatisfactory generalization towards different model architectures or cost enormous extra computation and memory. We proposeFedSpa, a novel PFL scheme that employs personalized sparse masks to customize sparse local models on the edge.
  arXiv  Detail & Related papers  (2022-01-27T08:43:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.