Accelerating Wireless Federated Learning via Nesterov's Momentum and Distributed Principle Component Analysis

• URL: http://arxiv.org/abs/2303.17885v1
• Date: Fri, 31 Mar 2023 08:41:42 GMT
• Title: Accelerating Wireless Federated Learning via Nesterov's Momentum and Distributed Principle Component Analysis
• Authors: Yanjie Dong, Luya Wang, Yuanfang Chi, Jia Wang, Haijun Zhang, Fei Richard Yu, Victor C. M. Leung, Xiping Hu
• Abstract summary: A wireless federated learning system is investigated by allowing a server and workers to exchange uncoded information via wireless channels. Since the workers frequently upload local to the server via bandwidth-limited channels, the uplink transmission from the workers to the server becomes a communication bottleneck. A one-shot distributed principle component analysis (PCA) is leveraged to reduce the dimension of the dimension of the communication bottleneck.
• Score: 59.127630388320036
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A wireless federated learning system is investigated by allowing a server and workers to exchange uncoded information via orthogonal wireless channels. Since the workers frequently upload local gradients to the server via bandwidth-limited channels, the uplink transmission from the workers to the server becomes a communication bottleneck. Therefore, a one-shot distributed principle component analysis (PCA) is leveraged to reduce the dimension of uploaded gradients such that the communication bottleneck is relieved. A PCA-based wireless federated learning (PCA-WFL) algorithm and its accelerated version (i.e., PCA-AWFL) are proposed based on the low-dimensional gradients and the Nesterov's momentum. For the non-convex loss functions, a finite-time analysis is performed to quantify the impacts of system hyper-parameters on the convergence of the PCA-WFL and PCA-AWFL algorithms. The PCA-AWFL algorithm is theoretically certified to converge faster than the PCA-WFL algorithm. Besides, the convergence rates of PCA-WFL and PCA-AWFL algorithms quantitatively reveal the linear speedup with respect to the number of workers over the vanilla gradient descent algorithm. Numerical results are used to demonstrate the improved convergence rates of the proposed PCA-WFL and PCA-AWFL algorithms over the benchmarks.

Related papers

• Predictive GAN-powered Multi-Objective Optimization for Hybrid Federated Split Learning [56.125720497163684]
  We propose a hybrid federated split learning framework in wireless networks. We design a parallel computing scheme for model splitting without label sharing, and theoretically analyze the influence of the delayed gradient caused by the scheme on the convergence speed.
  arXiv  Detail & Related papers  (2022-09-02T10:29:56Z)
• Unit-Modulus Wireless Federated Learning Via Penalty Alternating Minimization [64.76619508293966]
  Wireless federated learning (FL) is an emerging machine learning paradigm that trains a global parametric model from distributed datasets via wireless communications. This paper proposes a wireless FL framework, which uploads local model parameters and computes global model parameters via wireless communications.
  arXiv  Detail & Related papers  (2021-08-31T08:19:54Z)
• FAST-PCA: A Fast and Exact Algorithm for Distributed Principal Component Analysis [12.91948651812873]
  Principal Component Analysis (PCA) is a fundamental data preprocessing tool in the world of machine learning. This paper proposes a distributed PCA algorithm called FAST-PCA (Fast and exAct diSTributed PCA)
  arXiv  Detail & Related papers  (2021-08-27T16:10:59Z)
• Turning Channel Noise into an Accelerator for Over-the-Air Principal Component Analysis [65.31074639627226]
  Principal component analysis (PCA) is a technique for extracting the linear structure of a dataset. We propose the deployment of PCA over a multi-access channel based on the algorithm of gradient descent. Over-the-air aggregation is adopted to reduce the multi-access latency, giving the name over-the-air PCA.
  arXiv  Detail & Related papers  (2021-04-20T16:28:33Z)
• DeEPCA: Decentralized Exact PCA with Linear Convergence Rate [26.819982387028595]
  textttDe EPCA is the first decentralized PCA algorithm with the number of communication rounds for each power independent of target precision. Compared to existing algorithms, the proposed method is easier to tune in practice, with an improved overall communication cost.
  arXiv  Detail & Related papers  (2021-02-08T03:53:09Z)
• Federated Learning on the Road: Autonomous Controller Design for Connected and Autonomous Vehicles [109.71532364079711]
  A new federated learning (FL) framework is proposed for designing the autonomous controller of connected and autonomous vehicles (CAVs) A novel dynamic federated proximal (DFP) algorithm is proposed that accounts for the mobility of CAVs, the wireless fading channels, and the unbalanced and nonindependent and identically distributed data across CAVs. A rigorous convergence analysis is performed for the proposed algorithm to identify how fast the CAVs converge to using the optimal controller.
  arXiv  Detail & Related papers  (2021-02-05T19:57:47Z)
• Bayesian Federated Learning over Wireless Networks [87.37301441859925]
  Federated learning is a privacy-preserving and distributed training method using heterogeneous data sets stored at local devices. This paper presents an efficient modified BFL algorithm called scalableBFL (SBFL)
  arXiv  Detail & Related papers  (2020-12-31T07:32:44Z)
• Fast Convergence Algorithm for Analog Federated Learning [30.399830943617772]
  We propose an AirComp-based FedSplit algorithm for efficient analog federated learning over wireless channels. We prove that the proposed algorithm linearly converges to the optimal solutions under the assumption that the objective function is strongly convex and smooth. Our algorithm is theoretically and experimentally verified to be much more robust to the ill-conditioned problems with faster convergence compared with other benchmark FL algorithms.
  arXiv  Detail & Related papers  (2020-10-30T10:59:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.