Scalable Hierarchical Over-the-Air Federated Learning
- URL: http://arxiv.org/abs/2211.16162v3
- Date: Thu, 11 Jan 2024 12:47:47 GMT
- Title: Scalable Hierarchical Over-the-Air Federated Learning
- Authors: Seyed Mohammad Azimi-Abarghouyi, Viktoria Fodor
- Abstract summary: This work introduces a new two-level learning method designed to handle both interference and device data heterogeneity.
We present a comprehensive mathematical approach to derive the convergence bound for the proposed algorithm.
Despite the interference and data heterogeneity, the proposed algorithm achieves high learning accuracy for a variety of parameters.
- Score: 3.8798345704175534
- License: http://creativecommons.org/publicdomain/zero/1.0/
- Abstract: When implementing hierarchical federated learning over wireless networks,
scalability assurance and the ability to handle both interference and device
data heterogeneity are crucial. This work introduces a new two-level learning
method designed to address these challenges, along with a scalable over-the-air
aggregation scheme for the uplink and a bandwidth-limited broadcast scheme for
the downlink that efficiently use a single wireless resource. To provide
resistance against data heterogeneity, we employ gradient aggregations.
Meanwhile, the impact of uplink and downlink interference is minimized through
optimized receiver normalizing factors. We present a comprehensive mathematical
approach to derive the convergence bound for the proposed algorithm, applicable
to a multi-cluster wireless network encompassing any count of collaborating
clusters, and provide special cases and design remarks. As a key step to enable
a tractable analysis, we develop a spatial model for the setup by modeling
devices as a Poisson cluster process over the edge servers and rigorously
quantify uplink and downlink error terms due to the interference. Finally, we
show that despite the interference and data heterogeneity, the proposed
algorithm not only achieves high learning accuracy for a variety of parameters
but also significantly outperforms the conventional hierarchical learning
algorithm.
Related papers
- Robust Regression with Ensembles Communicating over Noisy Channels [16.344212996721346]
We study the problem of an ensemble of devices, implementing regression algorithms, that communicate through additive noisy channels.
We develop methods for optimizing the aggregation coefficients for the parameters of the noise in the channels, which can potentially be correlated.
Our results apply to the leading state-of-the-art ensemble regression methods: bagging and gradient boosting.
arXiv Detail & Related papers (2024-08-20T15:32:47Z) - LoRA-Ensemble: Efficient Uncertainty Modelling for Self-attention Networks [52.46420522934253]
We introduce LoRA-Ensemble, a parameter-efficient deep ensemble method for self-attention networks.
By employing a single pre-trained self-attention network with weights shared across all members, we train member-specific low-rank matrices for the attention projections.
Our method exhibits superior calibration compared to explicit ensembles and achieves similar or better accuracy across various prediction tasks and datasets.
arXiv Detail & Related papers (2024-05-23T11:10:32Z) - Hierarchical Over-the-Air Federated Learning with Awareness of
Interference and Data Heterogeneity [3.8798345704175534]
We introduce a scalable transmission scheme that efficiently uses a single wireless resource through over-the-air computation.
We show that despite the interference and the data heterogeneity, the proposed scheme achieves high learning accuracy and can significantly outperform the conventional hierarchical algorithm.
arXiv Detail & Related papers (2024-01-02T21:43:01Z) - Over-the-Air Federated Learning and Optimization [52.5188988624998]
We focus on Federated learning (FL) via edge-the-air computation (AirComp)
We describe the convergence of AirComp-based FedAvg (AirFedAvg) algorithms under both convex and non- convex settings.
For different types of local updates that can be transmitted by edge devices (i.e., model, gradient, model difference), we reveal that transmitting in AirFedAvg may cause an aggregation error.
In addition, we consider more practical signal processing schemes to improve the communication efficiency and extend the convergence analysis to different forms of model aggregation error caused by these signal processing schemes.
arXiv Detail & Related papers (2023-10-16T05:49:28Z) - Multi-agent Reinforcement Learning with Graph Q-Networks for Antenna
Tuning [60.94661435297309]
The scale of mobile networks makes it challenging to optimize antenna parameters using manual intervention or hand-engineered strategies.
We propose a new multi-agent reinforcement learning algorithm to optimize mobile network configurations globally.
We empirically demonstrate the performance of the algorithm on an antenna tilt tuning problem and a joint tilt and power control problem in a simulated environment.
arXiv Detail & Related papers (2023-01-20T17:06:34Z) - Personalized Decentralized Multi-Task Learning Over Dynamic
Communication Graphs [59.96266198512243]
We propose a decentralized and federated learning algorithm for tasks that are positively and negatively correlated.
Our algorithm uses gradients to calculate the correlations among tasks automatically, and dynamically adjusts the communication graph to connect mutually beneficial tasks and isolate those that may negatively impact each other.
We conduct experiments on a synthetic Gaussian dataset and a large-scale celebrity attributes (CelebA) dataset.
arXiv Detail & Related papers (2022-12-21T18:58:24Z) - Task-Oriented Sensing, Computation, and Communication Integration for
Multi-Device Edge AI [108.08079323459822]
This paper studies a new multi-intelligent edge artificial-latency (AI) system, which jointly exploits the AI model split inference and integrated sensing and communication (ISAC)
We measure the inference accuracy by adopting an approximate but tractable metric, namely discriminant gain.
arXiv Detail & Related papers (2022-07-03T06:57:07Z) - Over-the-Air Federated Learning via Second-Order Optimization [37.594140209854906]
Federated learning (FL) could result in task-oriented data traffic flows over wireless networks with limited radio resources.
We propose a novel over-the-air second-order federated optimization algorithm to simultaneously reduce the communication rounds and enable low-latency global model aggregation.
arXiv Detail & Related papers (2022-03-29T12:39:23Z) - Resource-constrained Federated Edge Learning with Heterogeneous Data:
Formulation and Analysis [8.863089484787835]
We propose a distributed approximate Newton-type Newton-type training scheme, namely FedOVA, to solve the heterogeneous statistical challenge brought by heterogeneous data.
FedOVA decomposes a multi-class classification problem into more straightforward binary classification problems and then combines their respective outputs using ensemble learning.
arXiv Detail & Related papers (2021-10-14T17:35:24Z) - A Compressive Sensing Approach for Federated Learning over Massive MIMO
Communication Systems [82.2513703281725]
Federated learning is a privacy-preserving approach to train a global model at a central server by collaborating with wireless devices.
We present a compressive sensing approach for federated learning over massive multiple-input multiple-output communication systems.
arXiv Detail & Related papers (2020-03-18T05:56:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.