Efficient and Reliable Overlay Networks for Decentralized Federated Learning

• URL: http://arxiv.org/abs/2112.15486v1
• Date: Sun, 12 Dec 2021 21:03:16 GMT
• Title: Efficient and Reliable Overlay Networks for Decentralized Federated Learning
• Authors: Yifan Hua, Kevin Miller, Andrea L. Bertozzi, Chen Qian, Bao Wang
• Abstract summary: We propose near-optimal overlay networks based on $d$-regular expander graphs to accelerate decentralized federated learning (DFL) In DFL a massive number of clients are connected by an overlay network, and they solve machine learning problems collaboratively without sharing raw data. We numerically verify the advantages of DFL with our proposed networks on various benchmark tasks.
• Score: 18.231702877235165
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: We propose near-optimal overlay networks based on $d$-regular expander graphs to accelerate decentralized federated learning (DFL) and improve its generalization. In DFL a massive number of clients are connected by an overlay network, and they solve machine learning problems collaboratively without sharing raw data. Our overlay network design integrates spectral graph theory and the theoretical convergence and generalization bounds for DFL. As such, our proposed overlay networks accelerate convergence, improve generalization, and enhance robustness to clients failures in DFL with theoretical guarantees. Also, we present an efficient algorithm to convert a given graph to a practical overlay network and maintaining the network topology after potential client failures. We numerically verify the advantages of DFL with our proposed networks on various benchmark tasks, ranging from image classification to language modeling using hundreds of clients.

Related papers

• Lightweight Federated Learning over Wireless Edge Networks [83.4818741890634]
  Federated (FL) is an alternative at network edge, but an alternative in wireless networks.<n>We derive a closed-form expression FL convergence gap transmission power, model pruning error, and quantization.<n> LTFL outperforms state-the-art schemes in experiments on real-world datasets.
  arXiv  Detail & Related papers  (2025-07-13T09:14:17Z)
• Overlay-based Decentralized Federated Learning in Bandwidth-limited Networks [3.9162099309900835]
  Decentralized federated learning (DFL) has the promise of boosting the deployment of artificial intelligence (AI) by directly learning across distributed agents without centralized coordination. Most existing solutions were based on the simplistic assumption that neighboring agents are physically adjacent in the underlying communication network. We jointly design the communication demands and the communication schedule for overlay-based DFL in bandwidth-limited networks without requiring explicit cooperation from the underlying network.
  arXiv  Detail & Related papers  (2024-08-08T18:05:11Z)
• Communication Efficient ConFederated Learning: An Event-Triggered SAGA Approach [67.27031215756121]
  Federated learning (FL) is a machine learning paradigm that targets model training without gathering the local data over various data sources. Standard FL, which employs a single server, can only support a limited number of users, leading to degraded learning capability. In this work, we consider a multi-server FL framework, referred to as emphConfederated Learning (CFL) in order to accommodate a larger number of users.
  arXiv  Detail & Related papers  (2024-02-28T03:27:10Z)
• Client Orchestration and Cost-Efficient Joint Optimization for NOMA-Enabled Hierarchical Federated Learning [55.49099125128281]
  We propose a non-orthogonal multiple access (NOMA) enabled HFL system under semi-synchronous cloud model aggregation. We show that the proposed scheme outperforms the considered benchmarks regarding HFL performance improvement and total cost reduction.
  arXiv  Detail & Related papers  (2023-11-03T13:34:44Z)
• Adaptive Federated Pruning in Hierarchical Wireless Networks [69.6417645730093]
  Federated Learning (FL) is a privacy-preserving distributed learning framework where a server aggregates models updated by multiple devices without accessing their private datasets. In this paper, we introduce model pruning for HFL in wireless networks to reduce the neural network scale. We show that our proposed HFL with model pruning achieves similar learning accuracy compared with the HFL without model pruning and reduces about 50 percent communication cost.
  arXiv  Detail & Related papers  (2023-05-15T22:04:49Z)
• Delay-Aware Hierarchical Federated Learning [7.292078085289465]
  The paper introduces delay-aware hierarchical federated learning (DFL) to improve the efficiency of distributed machine learning (ML) model training. During global synchronization, the cloud server consolidates local models with an outdated global model using a convex control algorithm. Numerical evaluations show DFL's superior performance in terms of faster global model, reduced convergence resource, and evaluations against communication delays.
  arXiv  Detail & Related papers  (2023-03-22T09:23:29Z)
• Hierarchical Personalized Federated Learning Over Massive Mobile Edge Computing Networks [95.39148209543175]
  We propose hierarchical PFL (HPFL), an algorithm for deploying PFL over massive MEC networks. HPFL combines the objectives of training loss minimization and round latency minimization while jointly determining the optimal bandwidth allocation.
  arXiv  Detail & Related papers  (2023-03-19T06:00:05Z)
• Federated Learning over Wireless IoT Networks with Optimized Communication and Resources [98.18365881575805]
  Federated learning (FL) as a paradigm of collaborative learning techniques has obtained increasing research attention. It is of interest to investigate fast responding and accurate FL schemes over wireless systems. We show that the proposed communication-efficient federated learning framework converges at a strong linear rate.
  arXiv  Detail & Related papers  (2021-10-22T13:25:57Z)
• FedFog: Network-Aware Optimization of Federated Learning over Wireless Fog-Cloud Systems [40.421253127588244]
  Federated learning (FL) is capable of performing large distributed machine learning tasks across multiple edge users by periodically aggregating trained local parameters. We first propose an efficient FL algorithm (called FedFog) to perform the local aggregation of gradient parameters at fog servers and global training update at the cloud.
  arXiv  Detail & Related papers  (2021-07-04T08:03:15Z)
• Coded Computing for Federated Learning at the Edge [3.385874614913973]
  Federated Learning (FL) enables training a global model from data generated locally at the client nodes, without moving client data to a centralized server. Recent work proposes to mitigate stragglers and speed up training for linear regression tasks by assigning redundant computations at the MEC server. We develop CodedFedL that addresses the difficult task of extending CFL to distributed non-linear regression and classification problems with multioutput labels.
  arXiv  Detail & Related papers  (2020-07-07T08:20:47Z)
• Network Adjustment: Channel Search Guided by FLOPs Utilization Ratio [101.84651388520584]
  This paper presents a new framework named network adjustment, which considers network accuracy as a function of FLOPs. Experiments on standard image classification datasets and a wide range of base networks demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2020-04-06T15:51:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.