Biased Over-the-Air Federated Learning under Wireless Heterogeneity

• URL: http://arxiv.org/abs/2403.19849v1
• Date: Thu, 28 Mar 2024 21:52:15 GMT
• Title: Biased Over-the-Air Federated Learning under Wireless Heterogeneity
• Authors: Muhammad Faraz Ul Abrar, Nicolò Michelusi,
• Abstract summary: We study the design of OTA device pre-scalers by focusing on the OTA-FL convergence. We identify two solutions of interest: minimum noise variance, and minimum noise variance zero-bias solutions.
• Score: 7.3716675761469945
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, Over-the-Air (OTA) computation has emerged as a promising federated learning (FL) paradigm that leverages the waveform superposition properties of the wireless channel to realize fast model updates. Prior work focused on the OTA device ``pre-scaler" design under \emph{homogeneous} wireless conditions, in which devices experience the same average path loss, resulting in zero-bias solutions. Yet, zero-bias designs are limited by the device with the worst average path loss and hence may perform poorly in \emph{heterogeneous} wireless settings. In this scenario, there may be a benefit in designing \emph{biased} solutions, in exchange for a lower variance in the model updates. To optimize this trade-off, we study the design of OTA device pre-scalers by focusing on the OTA-FL convergence. We derive an upper bound on the model ``optimality error", which explicitly captures the effect of bias and variance in terms of the choice of the pre-scalers. Based on this bound, we identify two solutions of interest: minimum noise variance, and minimum noise variance zero-bias solutions. Numerical evaluations show that using OTA device pre-scalers that minimize the variance of FL updates, while allowing a small bias, can provide high gains over existing schemes.

Related papers

• Biased Federated Learning under Wireless Heterogeneity [7.3716675761469945]
  Federated learning (FL) is a promising framework for computation, enabling collaborative model training without sharing private data. Existing wireless computation works primarily adopt two communication strategies: (1) over-the-air (OTA) which exploits wireless signal superposition, and (2) over-the-air (OTA) which allocates resources for convergence. This paper proposes novel OTA and digital FL updates that allow a structured, time-in-place bias, thereby reducing variance in FL updates.
  arXiv  Detail & Related papers  (2025-03-08T05:55:14Z)
• Rendering Wireless Environments Useful for Gradient Estimators: A Zero-Order Stochastic Federated Learning Method [14.986031916712108]
  Cross-device federated learning (FL) is a growing machine learning framework whereby multiple edge devices collaborate to train a model without disclosing their raw data. We show how to harness the wireless channel in the learning algorithm itself instead of to analyze it remove its impact.
  arXiv  Detail & Related papers  (2024-01-30T21:46:09Z)
• Over-the-Air Federated Learning and Optimization [52.5188988624998]
  We focus on Federated learning (FL) via edge-the-air computation (AirComp) We describe the convergence of AirComp-based FedAvg (AirFedAvg) algorithms under both convex and non- convex settings. For different types of local updates that can be transmitted by edge devices (i.e., model, gradient, model difference), we reveal that transmitting in AirFedAvg may cause an aggregation error. In addition, we consider more practical signal processing schemes to improve the communication efficiency and extend the convergence analysis to different forms of model aggregation error caused by these signal processing schemes.
  arXiv  Detail & Related papers  (2023-10-16T05:49:28Z)
• Channel and Gradient-Importance Aware Device Scheduling for Over-the-Air Federated Learning [31.966999085992505]
  Federated learning (FL) is a privacy-preserving distributed training scheme. We propose a device scheduling framework for over-the-air FL, named PO-FL, to mitigate the negative impact of channel noise distortion.
  arXiv  Detail & Related papers  (2023-05-26T12:04:59Z)
• Over-the-Air Federated Learning with Joint Adaptive Computation and Power Control [30.7130850260223]
  Over-the-air computation learning (OTA-FL) is considered in this paper. OTA-FL exploits the superposition property of the wireless medium and performs aggregation over the air for free.
  arXiv  Detail & Related papers  (2022-05-12T03:28:03Z)
• Deep Equilibrium Optical Flow Estimation [80.80992684796566]
  Recent state-of-the-art (SOTA) optical flow models use finite-step recurrent update operations to emulate traditional algorithms. These RNNs impose large computation and memory overheads, and are not directly trained to model such stable estimation. We propose deep equilibrium (DEQ) flow estimators, an approach that directly solves for the flow as the infinite-level fixed point of an implicit layer.
  arXiv  Detail & Related papers  (2022-04-18T17:53:44Z)
• Over-the-Air Federated Learning with Retransmissions (Extended Version) [21.37147806100865]
  We study the impact of estimation errors on the convergence of Federated Learning (FL) over resource-constrained wireless networks. We propose retransmissions as a method to improve FL convergence over resource-constrained wireless networks.
  arXiv  Detail & Related papers  (2021-11-19T15:17:15Z)
• Unit-Modulus Wireless Federated Learning Via Penalty Alternating Minimization [64.76619508293966]
  Wireless federated learning (FL) is an emerging machine learning paradigm that trains a global parametric model from distributed datasets via wireless communications. This paper proposes a wireless FL framework, which uploads local model parameters and computes global model parameters via wireless communications.
  arXiv  Detail & Related papers  (2021-08-31T08:19:54Z)
• Edge Federated Learning Via Unit-Modulus Over-The-Air Computation (Extended Version) [64.76619508293966]
  This paper proposes a unit-modulus over-the-air computation (UM-AirComp) framework to facilitate efficient edge federated learning. It uploads simultaneously local model parameters and updates global model parameters via analog beamforming. We demonstrate the implementation of UM-AirComp in a vehicle-to-everything autonomous driving simulation platform.
  arXiv  Detail & Related papers  (2021-01-28T15:10:22Z)
• Rate Region for Indirect Multiterminal Source Coding in Federated Learning [49.574683687858126]
  A large number of edge devices send their updates to the edge at each round of the local model. Existing works do not leverage the focus in the information transmitted by different edges. This paper studies the rate region for the indirect multiterminal source coding FL.
  arXiv  Detail & Related papers  (2021-01-21T16:17:39Z)
• Over-the-Air Federated Learning from Heterogeneous Data [107.05618009955094]
  Federated learning (FL) is a framework for distributed learning of centralized models. We develop a Convergent OTA FL (COTAF) algorithm which enhances the common local gradient descent (SGD) FL algorithm. We numerically show that the precoding induced by COTAF notably improves the convergence rate and the accuracy of models trained via OTA FL.
  arXiv  Detail & Related papers  (2020-09-27T08:28:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.