Non-Convex Over-the-Air Heterogeneous Federated Learning: A Bias-Variance Trade-off

• URL: http://arxiv.org/abs/2510.26722v3
• Date: Thu, 06 Nov 2025 17:41:05 GMT
• Title: Non-Convex Over-the-Air Heterogeneous Federated Learning: A Bias-Variance Trade-off
• Abstract summary: Over-the-air (OTA) convex learning (FL) has been well recognized as a scalable paradigm that exploits the superposition of the wireless multiple-access channel to model updates in a single use.<n>We develop novel OTAFL SGD updates that allow a structured bias, however, facilitating reduced gradient updates.
• Score: 6.832750677765499
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Over-the-air (OTA) federated learning (FL) has been well recognized as a scalable paradigm that exploits the waveform superposition of the wireless multiple-access channel to aggregate model updates in a single use. Existing OTA-FL designs largely enforce zero-bias model updates by either assuming \emph{homogeneous} wireless conditions (equal path loss across devices) or forcing zero-bias updates to guarantee convergence. Under \emph{heterogeneous} wireless scenarios, however, such designs are constrained by the weakest device and inflate the update variance. Moreover, prior analyses of biased OTA-FL largely address convex objectives, while most modern AI models are highly non-convex. Motivated by these gaps, we study OTA-FL with stochastic gradient descent (SGD) for general smooth non-convex objectives under wireless heterogeneity. We develop novel OTA-FL SGD updates that allow a structured, time-invariant model bias while facilitating reduced variance updates. We derive a finite-time stationarity bound (expected time average squared gradient norm) that explicitly reveals a bias-variance trade-off. To optimize this trade-off, we pose a non-convex joint OTA power-control design and develop an efficient successive convex approximation (SCA) algorithm that requires only statistical CSI at the base station. Experiments on a non-convex image classification task validate the approach: the SCA-based design accelerates convergence via an optimized bias and improves generalization over prior OTA-FL baselines.

Related papers

• ResAD: Normalized Residual Trajectory Modeling for End-to-End Autonomous Driving [64.42138266293202]
  ResAD is a Normalized Residual Trajectory Modeling framework.<n>It reframes the learning task to predict the residual deviation from an inertial reference.<n>On the NAVSIM benchmark, ResAD achieves a state-of-the-art PDMS of 88.6 using a vanilla diffusion policy.
  arXiv  Detail & Related papers  (2025-10-09T17:59:36Z)
• Harnessing Optimization Dynamics for Curvature-Informed Model Merging [17.42364575754576]
  In supervised fine-tuning, multiple capability-based SFT checkpoints must be consolidated into a single model.<n>We introduce Optimization Trajectory Aware (OTA) Merging and Fast Fisher Grafting (FFG)<n>OTA+FFG improves merged-model quality over strong weight-space baselines, reduces negative transfer, and remains robust across sparsity levels.
  arXiv  Detail & Related papers  (2025-09-14T08:59:53Z)
• Lightweight Federated Learning over Wireless Edge Networks [83.4818741890634]
  Federated (FL) is an alternative at network edge, but an alternative in wireless networks.<n>We derive a closed-form expression FL convergence gap transmission power, model pruning error, and quantization.<n> LTFL outperforms state-the-art schemes in experiments on real-world datasets.
  arXiv  Detail & Related papers  (2025-07-13T09:14:17Z)
• Biased Federated Learning under Wireless Heterogeneity [7.3716675761469945]
  Federated learning (FL) is a promising framework for computation, enabling collaborative model training without sharing private data.<n>Existing wireless computation works primarily adopt two communication strategies: (1) over-the-air (OTA) which exploits wireless signal superposition, and (2) over-the-air (OTA) which allocates resources for convergence.<n>This paper proposes novel OTA and digital FL updates that allow a structured, time-in-place bias, thereby reducing variance in FL updates.
  arXiv  Detail & Related papers  (2025-03-08T05:55:14Z)
• Biased Over-the-Air Federated Learning under Wireless Heterogeneity [7.3716675761469945]
  We study the design of OTA device pre-scalers by focusing on the OTA-FL convergence. We identify two solutions of interest: minimum noise variance, and minimum noise variance zero-bias solutions.
  arXiv  Detail & Related papers  (2024-03-28T21:52:15Z)
• Stragglers-Aware Low-Latency Synchronous Federated Learning via Layer-Wise Model Updates [71.81037644563217]
  Synchronous federated learning (FL) is a popular paradigm for collaborative edge learning. As some of the devices may have limited computational resources and varying availability, FL latency is highly sensitive to stragglers. We propose straggler-aware layer-wise federated learning (SALF) that leverages the optimization procedure of NNs via backpropagation to update the global model in a layer-wise fashion.
  arXiv  Detail & Related papers  (2024-03-27T09:14:36Z)
• FedNAR: Federated Optimization with Normalized Annealing Regularization [54.42032094044368]
  We explore the choices of weight decay and identify that weight decay value appreciably influences the convergence of existing FL algorithms. We develop Federated optimization with Normalized Annealing Regularization (FedNAR), a plug-in that can be seamlessly integrated into any existing FL algorithms.
  arXiv  Detail & Related papers  (2023-10-04T21:11:40Z)
• Over-the-Air Federated Learning with Joint Adaptive Computation and Power Control [30.7130850260223]
  Over-the-air computation learning (OTA-FL) is considered in this paper. OTA-FL exploits the superposition property of the wireless medium and performs aggregation over the air for free.
  arXiv  Detail & Related papers  (2022-05-12T03:28:03Z)
• Over-the-Air Federated Learning from Heterogeneous Data [107.05618009955094]
  Federated learning (FL) is a framework for distributed learning of centralized models. We develop a Convergent OTA FL (COTAF) algorithm which enhances the common local gradient descent (SGD) FL algorithm. We numerically show that the precoding induced by COTAF notably improves the convergence rate and the accuracy of models trained via OTA FL.
  arXiv  Detail & Related papers  (2020-09-27T08:28:25Z)
• Gradient Statistics Aware Power Control for Over-the-Air Federated Learning [59.40860710441232]
  Federated learning (FL) is a promising technique that enables many edge devices to train a machine learning model collaboratively in wireless networks. This paper studies the power control problem for over-the-air FL by taking gradient statistics into account.
  arXiv  Detail & Related papers  (2020-03-04T14:06:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.