FedCau: A Proactive Stop Policy for Communication and Computation Efficient Federated Learning

• URL: http://arxiv.org/abs/2204.07773v2
• Date: Tue, 26 Mar 2024 10:34:14 GMT
• Title: FedCau: A Proactive Stop Policy for Communication and Computation Efficient Federated Learning
• Authors: Afsaneh Mahmoudi, Hossein S. Ghadikolaei, José Mairton Barros Da Silva Júnior, Carlo Fischione,
• Abstract summary: We investigate efficient distributed training of a Federated Learning(FL) model over a wireless network of wireless devices. We show that, given a total cost budget, the training performance degrades as either the background communication traffic or the dimension of the training problem increases.
• Score: 9.107162745224162
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper investigates efficient distributed training of a Federated Learning~(FL) model over a wireless network of wireless devices. The communication iterations of the distributed training algorithm may be substantially deteriorated or even blocked by the effects of the devices' background traffic, packet losses, congestion, or latency. We abstract the communication-computation impacts as an `iteration cost' and propose a cost-aware causal FL algorithm~(FedCau) to tackle this problem. We propose an iteration-termination method that trade-offs the training performance and networking costs. We apply our approach when clients use the slotted-ALOHA, the carrier-sense multiple access with collision avoidance~(CSMA/CA), and the orthogonal frequency-division multiple access~(OFDMA) protocols. We show that, given a total cost budget, the training performance degrades as either the background communication traffic or the dimension of the training problem increases. Our results demonstrate the importance of proactively designing optimal cost-efficient stopping criteria to avoid unnecessary communication-computation costs to achieve only a marginal FL training improvement. We validate our method by training and testing FL over the MNIST dataset. Finally, we apply our approach to existing communication efficient FL methods from the literature, achieving further efficiency. We conclude that cost-efficient stopping criteria are essential for the success of practical FL over wireless networks.

Related papers

• Learning for Cross-Layer Resource Allocation in MEC-Aided Cell-Free Networks [71.30914500714262]
  Cross-layer resource allocation over mobile edge computing (MEC)-aided cell-free networks can sufficiently exploit the transmitting and computing resources to promote the data rate. Joint subcarrier allocation and beamforming optimization are investigated for the MEC-aided cell-free network from the perspective of deep learning.
  arXiv  Detail & Related papers  (2024-12-21T10:18:55Z)
• Latency Minimization for UAV-Enabled Federated Learning: Trajectory Design and Resource Allocation [47.20867891501245]
  Federated learning (FL) has become a transformative paradigm for distributed machine learning across wireless networks. We propose a novel framework that employs an unmanned aerial vehicle (UAV) as a mobile server to enhance the FL training process. We show that our proposed scheme not only surpasses existing benchmark schemes in reducing latency up to 15.29%, but also achieves training efficiency that nearly matches the ideal scenario.
  arXiv  Detail & Related papers  (2024-12-10T11:39:27Z)
• Digital Twin-Assisted Federated Learning with Blockchain in Multi-tier Computing Systems [67.14406100332671]
  In Industry 4.0 systems, resource-constrained edge devices engage in frequent data interactions. This paper proposes a digital twin (DT) and federated digital twin (FL) scheme. The efficacy of our proposed cooperative interference-based FL process has been verified through numerical analysis.
  arXiv  Detail & Related papers  (2024-11-04T17:48:02Z)
• Personalized Wireless Federated Learning for Large Language Models [75.22457544349668]
  Large language models (LLMs) have driven profound transformations in wireless networks.<n>Within wireless environments, the training of LLMs faces significant challenges related to security and privacy.<n>This paper presents a systematic analysis of the training stages of LLMs in wireless networks, including pre-training, instruction tuning, and alignment tuning.
  arXiv  Detail & Related papers  (2024-04-20T02:30:21Z)
• Performance Optimization for Variable Bitwidth Federated Learning in Wireless Networks [103.22651843174471]
  This paper considers improving wireless communication and computation efficiency in federated learning (FL) via model quantization. In the proposed bitwidth FL scheme, edge devices train and transmit quantized versions of their local FL model parameters to a coordinating server, which aggregates them into a quantized global model and synchronizes the devices. We show that the FL training process can be described as a Markov decision process and propose a model-based reinforcement learning (RL) method to optimize action selection over iterations.
  arXiv  Detail & Related papers  (2022-09-21T08:52:51Z)
• Resource Allocation for Compression-aided Federated Learning with High Distortion Rate [3.7530276852356645]
  We formulate an optimization-aided FL problem between the distortion rate, number of participating IoT devices, and convergence rate. By actively controlling participating IoT devices, we can avoid the training divergence of compression-aided FL while maintaining the communication efficiency.
  arXiv  Detail & Related papers  (2022-06-02T05:00:37Z)
• Towards Communication-Learning Trade-off for Federated Learning at the Network Edge [5.267288702335319]
  We propose a wireless learning (FL) system where network pruning is applied to local users with limited resources. Although beneficial to FL latency, it also deteriorates information loss.
  arXiv  Detail & Related papers  (2022-05-27T23:11:52Z)
• Over-the-Air Federated Learning with Retransmissions (Extended Version) [21.37147806100865]
  We study the impact of estimation errors on the convergence of Federated Learning (FL) over resource-constrained wireless networks. We propose retransmissions as a method to improve FL convergence over resource-constrained wireless networks.
  arXiv  Detail & Related papers  (2021-11-19T15:17:15Z)
• Federated Learning over Wireless IoT Networks with Optimized Communication and Resources [98.18365881575805]
  Federated learning (FL) as a paradigm of collaborative learning techniques has obtained increasing research attention. It is of interest to investigate fast responding and accurate FL schemes over wireless systems. We show that the proposed communication-efficient federated learning framework converges at a strong linear rate.
  arXiv  Detail & Related papers  (2021-10-22T13:25:57Z)
• ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Training [65.68511423300812]
  We propose ProgFed, a progressive training framework for efficient and effective federated learning. ProgFed inherently reduces computation and two-way communication costs while maintaining the strong performance of the final models. Our results show that ProgFed converges at the same rate as standard training on full models.
  arXiv  Detail & Related papers  (2021-10-11T14:45:00Z)
• Dynamic Attention-based Communication-Efficient Federated Learning [85.18941440826309]
  Federated learning (FL) offers a solution to train a global machine learning model. FL suffers performance degradation when client data distribution is non-IID. We propose a new adaptive training algorithm $textttAdaFL$ to combat this degradation.
  arXiv  Detail & Related papers  (2021-08-12T14:18:05Z)
• Convergence Analysis and System Design for Federated Learning over Wireless Networks [16.978276697446724]
  Federated learning (FL) has emerged as an important and promising learning scheme in IoT. FL training requires frequent model exchange, which is largely affected by the wireless communication network. In this paper, we analyze the convergence rate of FL training considering the joint impact of communication network and training settings.
  arXiv  Detail & Related papers  (2021-04-30T02:33:29Z)
• Delay Minimization for Federated Learning Over Wireless Communication Networks [172.42768672943365]
  The problem of delay computation for federated learning (FL) over wireless communication networks is investigated. A bisection search algorithm is proposed to obtain the optimal solution. Simulation results show that the proposed algorithm can reduce delay by up to 27.3% compared to conventional FL methods.
  arXiv  Detail & Related papers  (2020-07-05T19:00:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.