Accelerating Asynchronous Federated Learning Convergence via Opportunistic Mobile Relaying

• URL: http://arxiv.org/abs/2206.04742v2
• Date: Sun, 17 Mar 2024 16:38:47 GMT
• Title: Accelerating Asynchronous Federated Learning Convergence via Opportunistic Mobile Relaying
• Authors: Jieming Bian, Jie Xu,
• Abstract summary: We study the impact of mobility on the convergence performance of asynchronous Federated Learning (FL) algorithms. By exploiting mobility, the study shows that clients can indirectly communicate with the server through another client serving as a relay. We propose a new FL algorithm, called FedMobile, that incorporates opportunistic relaying and addresses key questions such as when and how to relay.
• Score: 3.802258033231335
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents a study on asynchronous Federated Learning (FL) in a mobile network setting. The majority of FL algorithms assume that communication between clients and the server is always available, however, this is not the case in many real-world systems. To address this issue, the paper explores the impact of mobility on the convergence performance of asynchronous FL. By exploiting mobility, the study shows that clients can indirectly communicate with the server through another client serving as a relay, creating additional communication opportunities. This enables clients to upload local model updates sooner or receive fresher global models. We propose a new FL algorithm, called FedMobile, that incorporates opportunistic relaying and addresses key questions such as when and how to relay. We prove that FedMobile achieves a convergence rate $O(\frac{1}{\sqrt{NT}})$, where $N$ is the number of clients and $T$ is the number of communication slots, and show that the optimal design involves an interesting trade-off on the best timing of relaying. The paper also presents an extension that considers data manipulation before relaying to reduce the cost and enhance privacy. Experiment results on a synthetic dataset and two real-world datasets verify our theoretical findings.

Related papers

• An Aggregation-Free Federated Learning for Tackling Data Heterogeneity [50.44021981013037]
  Federated Learning (FL) relies on the effectiveness of utilizing knowledge from distributed datasets. Traditional FL methods adopt an aggregate-then-adapt framework, where clients update local models based on a global model aggregated by the server from the previous training round. We introduce FedAF, a novel aggregation-free FL algorithm.
  arXiv  Detail & Related papers  (2024-04-29T05:55:23Z)
• Tunable Soft Prompts are Messengers in Federated Learning [55.924749085481544]
  Federated learning (FL) enables multiple participants to collaboratively train machine learning models using decentralized data sources. The lack of model privacy protection in FL becomes an unneglectable challenge. We propose a novel FL training approach that accomplishes information exchange among participants via tunable soft prompts.
  arXiv  Detail & Related papers  (2023-11-12T11:01:10Z)
• Towards Instance-adaptive Inference for Federated Learning [80.38701896056828]
  Federated learning (FL) is a distributed learning paradigm that enables multiple clients to learn a powerful global model by aggregating local training. In this paper, we present a novel FL algorithm, i.e., FedIns, to handle intra-client data heterogeneity by enabling instance-adaptive inference in the FL framework. Our experiments show that our FedIns outperforms state-of-the-art FL algorithms, e.g., a 6.64% improvement against the top-performing method with less than 15% communication cost on Tiny-ImageNet.
  arXiv  Detail & Related papers  (2023-08-11T09:58:47Z)
• Federated Learning for Semantic Parsing: Task Formulation, Evaluation Setup, New Algorithms [29.636944156801327]
  Multiple clients collaboratively train one global model without sharing their semantic parsing data. Lorar adjusts each client's contribution to the global model update based on its training loss reduction during each round. Clients with smaller datasets enjoy larger performance gains.
  arXiv  Detail & Related papers  (2023-05-26T19:25:49Z)
• Joint Client Assignment and UAV Route Planning for Indirect-Communication Federated Learning [20.541942109704987]
  A new framework called FedEx (Federated Learning via Model Express Delivery) is proposed. It employs mobile transporters, such as UAVs, to establish indirect communication channels between the server and clients. Two algorithms, FedEx-Sync and FedEx-Async, are proposed for synchronized and asynchronized learning at the transporter level.
  arXiv  Detail & Related papers  (2023-04-21T04:47:54Z)
• Federated Nearest Neighbor Machine Translation [66.8765098651988]
  In this paper, we propose a novel federated nearest neighbor (FedNN) machine translation framework. FedNN leverages one-round memorization-based interaction to share knowledge across different clients. Experiments show that FedNN significantly reduces computational and communication costs compared with FedAvg.
  arXiv  Detail & Related papers  (2023-02-23T18:04:07Z)
• Federated Learning via Indirect Server-Client Communications [20.541942109704987]
  Federated Learning (FL) is a communication-efficient and privacy-preserving distributed machine learning framework. We propose a novel FL framework, named FedEx, that utilizes mobile transporters to establish indirect communication channels between the server and the clients. Two algorithms, called FedEx-Sync and FedEx-Async, are developed depending on whether the transporters adopt a synchronized or an asynchronized schedule.
  arXiv  Detail & Related papers  (2023-02-14T20:12:36Z)
• Scalable Collaborative Learning via Representation Sharing [53.047460465980144]
  Federated learning (FL) and Split Learning (SL) are two frameworks that enable collaborative learning while keeping the data private (on device) In FL, each data holder trains a model locally and releases it to a central server for aggregation. In SL, the clients must release individual cut-layer activations (smashed data) to the server and wait for its response (during both inference and back propagation). In this work, we present a novel approach for privacy-preserving machine learning, where the clients collaborate via online knowledge distillation using a contrastive loss.
  arXiv  Detail & Related papers  (2022-11-20T10:49:22Z)
• DYNAFED: Tackling Client Data Heterogeneity with Global Dynamics [60.60173139258481]
  Local training on non-iid distributed data results in deflected local optimum. A natural solution is to gather all client data onto the server, such that the server has a global view of the entire data distribution. In this paper, we put forth an idea to collect and leverage global knowledge on the server without hindering data privacy.
  arXiv  Detail & Related papers  (2022-11-20T06:13:06Z)
• FedAR: Activity and Resource-Aware Federated Learning Model for Distributed Mobile Robots [1.332560004325655]
  A recently proposed Machine Learning algorithm called Federated Learning (FL) paves the path towards preserving data privacy. This paper proposes an FL model by monitoring client activities and leveraging available local computing resources. We consider such mobile robots as FL clients to understand their resource-constrained behavior in a real-world setting.
  arXiv  Detail & Related papers  (2021-01-11T05:27:37Z)
• RC-SSFL: Towards Robust and Communication-efficient Semi-supervised Federated Learning System [25.84191221776459]
  Federated Learning (FL) is an emerging decentralized artificial intelligence paradigm. Current systems rely heavily on a strong assumption: all clients have a wealth of ground truth labeled data. We present a practical Robust, and Communication-efficient Semi-supervised FL (RC-SSFL) system design.
  arXiv  Detail & Related papers  (2020-12-08T14:02:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.