Loosely Coupled Federated Learning Over Generative Models

• URL: http://arxiv.org/abs/2009.12999v1
• Date: Mon, 28 Sep 2020 01:09:23 GMT
• Title: Loosely Coupled Federated Learning Over Generative Models
• Authors: Shaoming Song, Yunfeng Shao, Jian Li
• Abstract summary: Federated learning (FL) was proposed to achieve collaborative machine learning among various clients without uploading private data. This paper proposes Loosely Coupled Federated Learning (LC-FL) to achieve low communication cost and heterogeneous federated learning.
• Score: 6.472716351335859
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated learning (FL) was proposed to achieve collaborative machine learning among various clients without uploading private data. However, due to model aggregation strategies, existing frameworks require strict model homogeneity, limiting the application in more complicated scenarios. Besides, the communication cost of FL's model and gradient transmission is extremely high. This paper proposes Loosely Coupled Federated Learning (LC-FL), a framework using generative models as transmission media to achieve low communication cost and heterogeneous federated learning. LC-FL can be applied on scenarios where clients possess different kinds of machine learning models. Experiments on real-world datasets covering different multiparty scenarios demonstrate the effectiveness of our proposal.

Related papers

• LanFL: Differentially Private Federated Learning with Large Language Models using Synthetic Samples [11.955062839855334]
  Federated Learning (FL) is a collaborative, privacy-preserving machine learning framework. The recent advent of powerful Large Language Models (LLMs) with tens to hundreds of billions of parameters makes the naive application of traditional FL methods impractical. This paper introduces a novel FL scheme for LLMs, named LanFL, which is purely prompt-based and treats the underlying LLMs as black boxes.
  arXiv  Detail & Related papers  (2024-10-24T19:28:33Z)
• FedPAE: Peer-Adaptive Ensemble Learning for Asynchronous and Model-Heterogeneous Federated Learning [9.084674176224109]
  Federated learning (FL) enables multiple clients with distributed data sources to collaboratively train a shared model without compromising data privacy. We introduce Federated Peer-Adaptive Ensemble Learning (FedPAE), a fully decentralized pFL algorithm that supports model heterogeneity and asynchronous learning. Our approach utilizes a peer-to-peer model sharing mechanism and ensemble selection to achieve a more refined balance between local and global information.
  arXiv  Detail & Related papers  (2024-10-17T22:47:19Z)
• Tunable Soft Prompts are Messengers in Federated Learning [55.924749085481544]
  Federated learning (FL) enables multiple participants to collaboratively train machine learning models using decentralized data sources. The lack of model privacy protection in FL becomes an unneglectable challenge. We propose a novel FL training approach that accomplishes information exchange among participants via tunable soft prompts.
  arXiv  Detail & Related papers  (2023-11-12T11:01:10Z)
• PFL-GAN: When Client Heterogeneity Meets Generative Models in Personalized Federated Learning [55.930403371398114]
  We propose a novel generative adversarial network (GAN) sharing and aggregation strategy for personalized learning (PFL) PFL-GAN addresses the client heterogeneity in different scenarios. More specially, we first learn the similarity among clients and then develop an weighted collaborative data aggregation. The empirical results through the rigorous experimentation on several well-known datasets demonstrate the effectiveness of PFL-GAN.
  arXiv  Detail & Related papers  (2023-08-23T22:38:35Z)
• Tensor Decomposition based Personalized Federated Learning [12.420951968273574]
  Federated learning (FL) is a new distributed machine learning framework that can achieve reliably collaborative training without collecting users' private data. Due to FL's frequent communication and average aggregation strategy, they experience challenges scaling to statistical diversity data and large-scale models. We propose a personalized FL framework, named Decomposition based Personalized learning (TDPFed), in which we design a novel tensorized local model with tensorized linear layers and convolutional layers to reduce the communication cost.
  arXiv  Detail & Related papers  (2022-08-27T08:09:14Z)
• FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learning [87.08902493524556]
  Federated learning(FL) has recently attracted increasing attention from academia and industry. We propose FedDM to build the global training objective from multiple local surrogate functions. In detail, we construct synthetic sets of data on each client to locally match the loss landscape from original data.
  arXiv  Detail & Related papers  (2022-07-20T04:55:18Z)
• Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customization [107.72786199113183]
  Federated learning (FL) provides a distributed learning framework for multiple participants to collaborate learning without sharing raw data. In this paper, we propose a novel Split-Mix FL strategy for heterogeneous participants that, once training is done, provides in-situ customization of model sizes and robustness.
  arXiv  Detail & Related papers  (2022-03-18T04:58:34Z)
• FedHM: Efficient Federated Learning for Heterogeneous Models via Low-rank Factorization [16.704006420306353]
  A scalable federated learning framework should address heterogeneous clients equipped with different computation and communication capabilities. This paper proposes FedHM, a novel federated model compression framework that distributes the heterogeneous low-rank models to clients and then aggregates them into a global full-rank model. Our solution enables the training of heterogeneous local models with varying computational complexities and aggregates a single global model.
  arXiv  Detail & Related papers  (2021-11-29T16:11:09Z)
• Edge-assisted Democratized Learning Towards Federated Analytics [67.44078999945722]
  We show the hierarchical learning structure of the proposed edge-assisted democratized learning mechanism, namely Edge-DemLearn. We also validate Edge-DemLearn as a flexible model training mechanism to build a distributed control and aggregation methodology in regions.
  arXiv  Detail & Related papers  (2020-12-01T11:46:03Z)
• HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients [42.365530133003816]
  We propose a new federated learning framework named HeteroFL to address heterogeneous clients equipped with very different computation and communication capabilities. Our solution can enable the training of heterogeneous local models with varying complexities. We show that adaptively distributing data according to clients' capabilities is both computation and communication efficient.
  arXiv  Detail & Related papers  (2020-10-03T02:55:33Z)
• Ensemble Distillation for Robust Model Fusion in Federated Learning [72.61259487233214]
  Federated Learning (FL) is a machine learning setting where many devices collaboratively train a machine learning model. In most of the current training schemes the central model is refined by averaging the parameters of the server model and the updated parameters from the client side. We propose ensemble distillation for model fusion, i.e. training the central classifier through unlabeled data on the outputs of the models from the clients.
  arXiv  Detail & Related papers  (2020-06-12T14:49:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.