Rendering Wireless Environments Useful for Gradient Estimators: A Zero-Order Stochastic Federated Learning Method

• URL: http://arxiv.org/abs/2401.17460v2
• Date: Tue, 23 Jul 2024 15:14:08 GMT
• Title: Rendering Wireless Environments Useful for Gradient Estimators: A Zero-Order Stochastic Federated Learning Method
• Authors: Elissa Mhanna, Mohamad Assaad,
• Abstract summary: Cross-device federated learning (FL) is a growing machine learning framework whereby multiple edge devices collaborate to train a model without disclosing their raw data. We show how to harness the wireless channel in the learning algorithm itself instead of to analyze it remove its impact.
• Score: 14.986031916712108
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Cross-device federated learning (FL) is a growing machine learning setting whereby multiple edge devices collaborate to train a model without disclosing their raw data. With the great number of mobile devices participating in more FL applications via the wireless environment, the practical implementation of these applications will be hindered due to the limited uplink capacity of devices, causing critical bottlenecks. In this work, we propose a novel doubly communication-efficient zero-order (ZO) method with a one-point gradient estimator that replaces communicating long vectors with scalar values and that harnesses the nature of the wireless communication channel, overcoming the need to know the channel state coefficient. It is the first method that includes the wireless channel in the learning algorithm itself instead of wasting resources to analyze it and remove its impact. We then offer a thorough analysis of the proposed zero-order federated learning (ZOFL) framework and prove that our method converges \textit{almost surely}, which is a novel result in nonconvex ZO optimization. We further prove a convergence rate of $O(\frac{1}{\sqrt[3]{K}})$ in the nonconvex setting. We finally demonstrate the potential of our algorithm with experimental results.

Related papers

• Communication and Energy Efficient Federated Learning using Zero-Order Optimization Technique [14.986031916712108]
  Federated learning (FL) is a popular machine learning technique that enables multiple users to collaboratively train a model while maintaining the user data privacy. A significant challenge in FL is the communication bottleneck in the upload direction, and thus the corresponding energy consumption of the devices. We show the superiority of our method, in terms of communication overhead and energy, as compared to standard gradient-based FL methods.
  arXiv  Detail & Related papers  (2024-09-24T20:57:22Z)
• Communication Efficient ConFederated Learning: An Event-Triggered SAGA Approach [67.27031215756121]
  Federated learning (FL) is a machine learning paradigm that targets model training without gathering the local data over various data sources. Standard FL, which employs a single server, can only support a limited number of users, leading to degraded learning capability. In this work, we consider a multi-server FL framework, referred to as emphConfederated Learning (CFL) in order to accommodate a larger number of users.
  arXiv  Detail & Related papers  (2024-02-28T03:27:10Z)
• Over-the-Air Federated Learning and Optimization [52.5188988624998]
  We focus on Federated learning (FL) via edge-the-air computation (AirComp) We describe the convergence of AirComp-based FedAvg (AirFedAvg) algorithms under both convex and non- convex settings. For different types of local updates that can be transmitted by edge devices (i.e., model, gradient, model difference), we reveal that transmitting in AirFedAvg may cause an aggregation error. In addition, we consider more practical signal processing schemes to improve the communication efficiency and extend the convergence analysis to different forms of model aggregation error caused by these signal processing schemes.
  arXiv  Detail & Related papers  (2023-10-16T05:49:28Z)
• Semi-Federated Learning: Convergence Analysis and Optimization of A Hybrid Learning Framework [70.83511997272457]
  We propose a semi-federated learning (SemiFL) paradigm to leverage both the base station (BS) and devices for a hybrid implementation of centralized learning (CL) and FL. We propose a two-stage algorithm to solve this intractable problem, in which we provide the closed-form solutions to the beamformers.
  arXiv  Detail & Related papers  (2023-10-04T03:32:39Z)
• Communication-Efficient Device Scheduling for Federated Learning Using Stochastic Optimization [26.559267845906746]
  Time learning (FL) is a useful tool in distributed machine learning that utilizes users' local datasets in a privacy-preserving manner. In this paper, we provide a novel convergence analysis non-efficient convergence bound algorithm. We also develop a new selection and power allocation algorithm that minimizes a function of the convergence bound and the average communication under a power constraint.
  arXiv  Detail & Related papers  (2022-01-19T23:25:24Z)
• Unit-Modulus Wireless Federated Learning Via Penalty Alternating Minimization [64.76619508293966]
  Wireless federated learning (FL) is an emerging machine learning paradigm that trains a global parametric model from distributed datasets via wireless communications. This paper proposes a wireless FL framework, which uploads local model parameters and computes global model parameters via wireless communications.
  arXiv  Detail & Related papers  (2021-08-31T08:19:54Z)
• Multi-task Federated Edge Learning (MtFEEL) in Wireless Networks [1.9250873974729816]
  Federated Learning (FL) has evolved as a promising technique to handle distributed machine learning across edge devices. A novel communication efficient FL algorithm for personalised learning in a wireless setting with guarantees is presented.
  arXiv  Detail & Related papers  (2021-08-05T10:54:38Z)
• Fast Federated Learning in the Presence of Arbitrary Device Unavailability [26.368873771739715]
  Federated Learning (FL) coordinates heterogeneous devices to collaboratively train a shared model while preserving user privacy. One challenge arises when devices drop out of the training process beyond the central server. We propose Im Federated Apatientaging (MIFA) to solve this problem.
  arXiv  Detail & Related papers  (2021-06-08T07:46:31Z)
• A Compressive Sensing Approach for Federated Learning over Massive MIMO Communication Systems [82.2513703281725]
  Federated learning is a privacy-preserving approach to train a global model at a central server by collaborating with wireless devices. We present a compressive sensing approach for federated learning over massive multiple-input multiple-output communication systems.
  arXiv  Detail & Related papers  (2020-03-18T05:56:27Z)
• Gradient Statistics Aware Power Control for Over-the-Air Federated Learning [59.40860710441232]
  Federated learning (FL) is a promising technique that enables many edge devices to train a machine learning model collaboratively in wireless networks. This paper studies the power control problem for over-the-air FL by taking gradient statistics into account.
  arXiv  Detail & Related papers  (2020-03-04T14:06:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.