Convergence of Federated Learning over a Noisy Downlink

• URL: http://arxiv.org/abs/2008.11141v1
• Date: Tue, 25 Aug 2020 16:15:05 GMT
• Title: Convergence of Federated Learning over a Noisy Downlink
• Authors: Mohammad Mohammadi Amiri, Deniz Gunduz, Sanjeev R. Kulkarni, H. Vincent Poor
• Abstract summary: We study federated learning, where power-limited wireless devices utilize their local datasets to collaboratively train a global model with the help of a remote parameter server. This framework requires downlink transmission from the PS to the devices and uplink transmission from the devices to the PS. The goal of this study is to investigate the impact of the bandwidth-limited shared wireless medium in both the downlink and uplink on the performance of FL.
• Score: 84.55126371346452
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study federated learning (FL), where power-limited wireless devices utilize their local datasets to collaboratively train a global model with the help of a remote parameter server (PS). The PS has access to the global model and shares it with the devices for local training, and the devices return the result of their local updates to the PS to update the global model. This framework requires downlink transmission from the PS to the devices and uplink transmission from the devices to the PS. The goal of this study is to investigate the impact of the bandwidth-limited shared wireless medium in both the downlink and uplink on the performance of FL with a focus on the downlink. To this end, the downlink and uplink channels are modeled as fading broadcast and multiple access channels, respectively, both with limited bandwidth. For downlink transmission, we first introduce a digital approach, where a quantization technique is employed at the PS to broadcast the global model update at a common rate such that all the devices can decode it. Next, we propose analog downlink transmission, where the global model is broadcast by the PS in an uncoded manner. We consider analog transmission over the uplink in both cases. We further analyze the convergence behavior of the proposed analog approach assuming that the uplink transmission is error-free. Numerical experiments show that the analog downlink approach provides significant improvement over the digital one, despite a significantly lower transmit power at the PS. The experimental results corroborate the convergence results, and show that a smaller number of local iterations should be used when the data distribution is more biased, and also when the devices have a better estimate of the global model in the analog downlink approach.

Related papers

• Time-Series JEPA for Predictive Remote Control under Capacity-Limited Networks [31.408649975934008]
  Time-Series Joint Embedding Predictive Architecture (TSEPA) and semantic actor trained through self-supervised learning. We propose a Time-Series Joint Embedding Predictive Architecture (TSEPA) and a semantic actor trained through self-supervised learning.
  arXiv  Detail & Related papers  (2024-06-07T11:35:15Z)
• Over-the-Air Federated Learning and Optimization [52.5188988624998]
  We focus on Federated learning (FL) via edge-the-air computation (AirComp) We describe the convergence of AirComp-based FedAvg (AirFedAvg) algorithms under both convex and non- convex settings. For different types of local updates that can be transmitted by edge devices (i.e., model, gradient, model difference), we reveal that transmitting in AirFedAvg may cause an aggregation error. In addition, we consider more practical signal processing schemes to improve the communication efficiency and extend the convergence analysis to different forms of model aggregation error caused by these signal processing schemes.
  arXiv  Detail & Related papers  (2023-10-16T05:49:28Z)
• Digital Over-the-Air Federated Learning in Multi-Antenna Systems [30.137208705209627]
  We study the performance optimization of federated learning (FL) over a realistic wireless communication system with digital modulation and over-the-air computation (AirComp) We propose a modified federated averaging (FedAvg) algorithm that combines digital modulation with AirComp to mitigate wireless fading while ensuring the communication efficiency. An artificial neural network (ANN) is used to estimate the local FL models of all devices and adjust the beamforming matrices at the PS for future model transmission.
  arXiv  Detail & Related papers  (2023-02-04T07:26:06Z)
• Federated Learning with Downlink Device Selection [92.14944020945846]
  We study federated edge learning, where a global model is trained collaboratively using privacy-sensitive data at the edge of a wireless network. A parameter server (PS) keeps track of the global model and shares it with the wireless edge devices for training using their private local data. We consider device selection based on downlink channels over which the PS shares the global model with the devices.
  arXiv  Detail & Related papers  (2021-07-07T22:42:39Z)
• Federated Learning over Wireless Device-to-Device Networks: Algorithms and Convergence Analysis [46.76179091774633]
  This paper studies federated learning (FL) over wireless device-to-device (D2D) networks. First, we introduce generic digital and analog wireless implementations of communication-efficient DSGD algorithms. Second, under the assumptions of convexity and connectivity, we provide convergence bounds for both implementations.
  arXiv  Detail & Related papers  (2021-01-29T17:42:26Z)
• Federated Learning over Noisy Channels: Convergence Analysis and Design Examples [17.89437720094451]
  Federated Learning (FL) works when both uplink and downlink communications have errors. How much communication noise can FL handle and what is its impact to the learning performance? This work is devoted to answering these practically important questions by explicitly incorporating both uplink and downlink noisy channels in the FL pipeline.
  arXiv  Detail & Related papers  (2021-01-06T18:57:39Z)
• Over-the-Air Federated Learning from Heterogeneous Data [107.05618009955094]
  Federated learning (FL) is a framework for distributed learning of centralized models. We develop a Convergent OTA FL (COTAF) algorithm which enhances the common local gradient descent (SGD) FL algorithm. We numerically show that the precoding induced by COTAF notably improves the convergence rate and the accuracy of models trained via OTA FL.
  arXiv  Detail & Related papers  (2020-09-27T08:28:25Z)
• Federated Learning With Quantized Global Model Updates [84.55126371346452]
  We study federated learning, which enables mobile devices to utilize their local datasets to train a global model. We introduce a lossy FL (LFL) algorithm, in which both the global model and the local model updates are quantized before being transmitted.
  arXiv  Detail & Related papers  (2020-06-18T16:55:20Z)
• Convergence of Update Aware Device Scheduling for Federated Learning at the Wireless Edge [84.55126371346452]
  We study federated learning at the wireless edge, where power-limited devices with local datasets collaboratively train a joint model with the help of a remote parameter server (PS) We design novel scheduling and resource allocation policies that decide on the subset of the devices to transmit at each round. The results of numerical experiments show that the proposed scheduling policy, based on both the channel conditions and the significance of the local model updates, provides a better long-term performance than scheduling policies based only on either of the two metrics individually.
  arXiv  Detail & Related papers  (2020-01-28T15:15:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.