A Multi-Token Coordinate Descent Method for Semi-Decentralized Vertical Federated Learning

• URL: http://arxiv.org/abs/2309.09977v1
• Date: Mon, 18 Sep 2023 17:59:01 GMT
• Title: A Multi-Token Coordinate Descent Method for Semi-Decentralized Vertical Federated Learning
• Authors: Pedro Valdeira, Yuejie Chi, Cl\'audia Soares, Jo\~ao Xavier
• Abstract summary: Communication efficiency is a major challenge in learning (FL) We propose Multi-Token Coordinate Descent (MTCD) MTCD is a tunable communication-efficient for semi-decentralized vertical federation setups.
• Score: 24.60603310894048
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Communication efficiency is a major challenge in federated learning (FL). In client-server schemes, the server constitutes a bottleneck, and while decentralized setups spread communications, they do not necessarily reduce them due to slower convergence. We propose Multi-Token Coordinate Descent (MTCD), a communication-efficient algorithm for semi-decentralized vertical federated learning, exploiting both client-server and client-client communications when each client holds a small subset of features. Our multi-token method can be seen as a parallel Markov chain (block) coordinate descent algorithm and it subsumes the client-server and decentralized setups as special cases. We obtain a convergence rate of $\mathcal{O}(1/T)$ for nonconvex objectives when tokens roam over disjoint subsets of clients and for convex objectives when they roam over possibly overlapping subsets. Numerical results show that MTCD improves the state-of-the-art communication efficiency and allows for a tunable amount of parallel communications.

Related papers

• FedScalar: A Communication efficient Federated Learning [0.0]
  Federated learning (FL) has gained considerable popularity for distributed machine learning. emphFedScalar enables agents to communicate updates using a single scalar.
  arXiv  Detail & Related papers  (2024-10-03T07:06:49Z)
• Communication Efficient ConFederated Learning: An Event-Triggered SAGA Approach [67.27031215756121]
  Federated learning (FL) is a machine learning paradigm that targets model training without gathering the local data over various data sources. Standard FL, which employs a single server, can only support a limited number of users, leading to degraded learning capability. In this work, we consider a multi-server FL framework, referred to as emphConfederated Learning (CFL) in order to accommodate a larger number of users.
  arXiv  Detail & Related papers  (2024-02-28T03:27:10Z)
• Scheduling and Communication Schemes for Decentralized Federated Learning [0.31410859223862103]
  A decentralized federated learning (DFL) model with the gradient descent (SGD) algorithm has been introduced. Three scheduling policies for DFL have been proposed for communications between the clients and the parallel servers. Results show that the proposed scheduling polices have an impact both on the speed of convergence and in the final global model.
  arXiv  Detail & Related papers  (2023-11-27T17:35:28Z)
• Communication-Efficient Decentralized Federated Learning via One-Bit Compressive Sensing [52.402550431781805]
  Decentralized federated learning (DFL) has gained popularity due to its practicality across various applications. Compared to the centralized version, training a shared model among a large number of nodes in DFL is more challenging. We develop a novel algorithm based on the framework of the inexact alternating direction method (iADM)
  arXiv  Detail & Related papers  (2023-08-31T12:22:40Z)
• Efficient Distribution Similarity Identification in Clustered Federated Learning via Principal Angles Between Client Data Subspaces [59.33965805898736]
  Clustered learning has been shown to produce promising results by grouping clients into clusters. Existing FL algorithms are essentially trying to group clients together with similar distributions. Prior FL algorithms attempt similarities indirectly during training.
  arXiv  Detail & Related papers  (2022-09-21T17:37:54Z)
• DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training [84.81043932706375]
  We propose a novel personalized federated learning framework in a decentralized (peer-to-peer) communication protocol named Dis-PFL. Dis-PFL employs personalized sparse masks to customize sparse local models on the edge. We demonstrate that our method can easily adapt to heterogeneous local clients with varying computation complexities.
  arXiv  Detail & Related papers  (2022-06-01T02:20:57Z)
• Cross-Silo Federated Learning for Multi-Tier Networks with Vertical and Horizontal Data Partitioning [20.410494162333972]
  Tiered Decentralized Coordinate Descent (TDCD) is a decentralized training algorithm for two-tiered communication networks. We present a theoretical analysis of our algorithm and show the dependence of the convergence rate on the number of vertical partitions and the number of local updates.
  arXiv  Detail & Related papers  (2021-08-19T22:01:04Z)
• Federated Multi-Target Domain Adaptation [99.93375364579484]
  Federated learning methods enable us to train machine learning models on distributed user data while preserving its privacy. We consider a more practical scenario where the distributed client data is unlabeled, and a centralized labeled dataset is available on the server. We propose an effective DualAdapt method to address the new challenges.
  arXiv  Detail & Related papers  (2021-08-17T17:53:05Z)
• Decentralized Federated Averaging [17.63112147669365]
  Federated averaging (FedAvg) is a communication efficient algorithm for the distributed training with an enormous number of clients. We study the decentralized FedAvg with momentum (DFedAvgM), which is implemented on clients that are connected by an undirected graph.
  arXiv  Detail & Related papers  (2021-04-23T02:01:30Z)
• Multi-Tier Federated Learning for Vertically Partitioned Data [2.817412580574242]
  We consider decentralized model training in tiered datasets networks. Our network model consists of a set of silos, each holding a vertical partition of the data.
  arXiv  Detail & Related papers  (2021-02-06T17:34:14Z)
• Faster Non-Convex Federated Learning via Global and Local Momentum [57.52663209739171]
  textttFedGLOMO is the first (first-order) FLtexttFedGLOMO algorithm. Our algorithm is provably optimal even with communication between the clients and the server.
  arXiv  Detail & Related papers  (2020-12-07T21:05:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.