Decentralized Federated Learning Over Imperfect Communication Channels

• URL: http://arxiv.org/abs/2405.12894v1
• Date: Tue, 21 May 2024 16:04:32 GMT
• Title: Decentralized Federated Learning Over Imperfect Communication Channels
• Authors: Weicai Li, Tiejun Lv, Wei Ni, Jingbo Zhao, Ekram Hossain, H. Vincent Poor,
• Abstract summary: This paper analyzes the impact of imperfect communication channels on decentralized federated learning (D-FL) It determines the optimal number of local aggregations per training round, adapting to the network topology and imperfect channels. It is seen that D-FL, with an optimal number of local aggregations, can outperform its potential alternatives by over 10% in training accuracy.
• Score: 68.08499874460857
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper analyzes the impact of imperfect communication channels on decentralized federated learning (D-FL) and subsequently determines the optimal number of local aggregations per training round, adapting to the network topology and imperfect channels. We start by deriving the bias of locally aggregated D-FL models under imperfect channels from the ideal global models requiring perfect channels and aggregations. The bias reveals that excessive local aggregations can accumulate communication errors and degrade convergence. Another important aspect is that we analyze a convergence upper bound of D-FL based on the bias. By minimizing the bound, the optimal number of local aggregations is identified to balance a trade-off with accumulation of communication errors in the absence of knowledge of the channels. With this knowledge, the impact of communication errors can be alleviated, allowing the convergence upper bound to decrease throughout aggregations. Experiments validate our convergence analysis and also identify the optimal number of local aggregations on two widely considered image classification tasks. It is seen that D-FL, with an optimal number of local aggregations, can outperform its potential alternatives by over 10% in training accuracy.

Related papers

• Over-the-Air Federated Learning and Optimization [52.5188988624998]
  We focus on Federated learning (FL) via edge-the-air computation (AirComp) We describe the convergence of AirComp-based FedAvg (AirFedAvg) algorithms under both convex and non- convex settings. For different types of local updates that can be transmitted by edge devices (i.e., model, gradient, model difference), we reveal that transmitting in AirFedAvg may cause an aggregation error. In addition, we consider more practical signal processing schemes to improve the communication efficiency and extend the convergence analysis to different forms of model aggregation error caused by these signal processing schemes.
  arXiv  Detail & Related papers  (2023-10-16T05:49:28Z)
• Semi-Federated Learning: Convergence Analysis and Optimization of A Hybrid Learning Framework [70.83511997272457]
  We propose a semi-federated learning (SemiFL) paradigm to leverage both the base station (BS) and devices for a hybrid implementation of centralized learning (CL) and FL. We propose a two-stage algorithm to solve this intractable problem, in which we provide the closed-form solutions to the beamformers.
  arXiv  Detail & Related papers  (2023-10-04T03:32:39Z)
• Integrating Local Real Data with Global Gradient Prototypes for Classifier Re-Balancing in Federated Long-Tailed Learning [60.41501515192088]
  Federated Learning (FL) has become a popular distributed learning paradigm that involves multiple clients training a global model collaboratively. The data samples usually follow a long-tailed distribution in the real world, and FL on the decentralized and long-tailed data yields a poorly-behaved global model. In this work, we integrate the local real data with the global gradient prototypes to form the local balanced datasets.
  arXiv  Detail & Related papers  (2023-01-25T03:18:10Z)
• Characterization of the Global Bias Problem in Aerial Federated Learning [30.555678487200456]
  Unmanned aerial vehicles (UAVs) mobility enables flexible and customized federated learning (FL) at the network edge. The distribution of the global model and the aggregation of the local updates within the FL learning rounds at the UAVs are governed by the reliability of the wireless channel. This paper characterizes the global bias problem of aerial FL in large-scale UAV networks.
  arXiv  Detail & Related papers  (2022-12-29T16:19:36Z)
• Fundamental Limits of Communication Efficiency for Model Aggregation in Distributed Learning: A Rate-Distortion Approach [54.311495894129585]
  We study the limit of communication cost of model aggregation in distributed learning from a rate-distortion perspective. It is found that the communication gain by exploiting the correlation between worker nodes is significant for SignSGD.
  arXiv  Detail & Related papers  (2022-06-28T13:10:40Z)
• Disentangled Federated Learning for Tackling Attributes Skew via Invariant Aggregation and Diversity Transferring [104.19414150171472]
  Attributes skews the current federated learning (FL) frameworks from consistent optimization directions among the clients. We propose disentangled federated learning (DFL) to disentangle the domain-specific and cross-invariant attributes into two complementary branches. Experiments verify that DFL facilitates FL with higher performance, better interpretability, and faster convergence rate, compared with SOTA FL methods.
  arXiv  Detail & Related papers  (2022-06-14T13:12:12Z)
• Revisiting Communication-Efficient Federated Learning with Balanced Global and Local Updates [14.851898446967672]
  We investigate and analyze the optimal trade-off between the number of local trainings and that of global aggregations. Our proposed scheme can achieve a better performance in terms of the prediction accuracy, and converge much faster than the baseline schemes.
  arXiv  Detail & Related papers  (2022-05-03T13:05:26Z)
• Decentralized Event-Triggered Federated Learning with Heterogeneous Communication Thresholds [12.513477328344255]
  We propose a novel methodology for distributed model aggregations via asynchronous, event-triggered consensus iterations over a network graph topology. We demonstrate that our methodology achieves the globally optimal learning model under standard assumptions in distributed learning and graph consensus literature.
  arXiv  Detail & Related papers  (2022-04-07T20:35:37Z)
• On Convergence of Federated Averaging Langevin Dynamics [22.013125418713763]
  We propose a federated averaging Langevin algorithm (FA-LD) for uncertainty quantification and mean predictions with distributed clients. We develop theoretical guarantees for FA-LD for strongly log-con distributions with non-icaved data. We show convergence results based on different averaging schemes where only partial device updates are available.
  arXiv  Detail & Related papers  (2021-12-09T18:54:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.