EFFGAN: Ensembles of fine-tuned federated GANs
- URL: http://arxiv.org/abs/2206.11682v1
- Date: Thu, 23 Jun 2022 13:12:16 GMT
- Title: EFFGAN: Ensembles of fine-tuned federated GANs
- Authors: Ebba Ekblom, Edvin Listo Zec, Olof Mogren
- Abstract summary: We study the task of how to learn a data distribution when training data is heterogeneously decentralized over clients and cannot be shared.
Our goal is to sample from this distribution centrally, while the data never leaves the clients.
Being an ensemble of local expert generators, EFFGAN is able to learn the data distribution over all clients and mitigate client drift.
- Score: 0.39594431485015086
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Generative adversarial networks have proven to be a powerful tool for
learning complex and high-dimensional data distributions, but issues such as
mode collapse have been shown to make it difficult to train them. This is an
even harder problem when the data is decentralized over several clients in a
federated learning setup, as problems such as client drift and non-iid data
make it hard for federated averaging to converge.
In this work, we study the task of how to learn a data distribution when
training data is heterogeneously decentralized over clients and cannot be
shared. Our goal is to sample from this distribution centrally, while the data
never leaves the clients. We show using standard benchmark image datasets that
existing approaches fail in this setting, experiencing so-called client drift
when the local number of epochs becomes to large. We thus propose a novel
approach we call EFFGAN: Ensembles of fine-tuned federated GANs. Being an
ensemble of local expert generators, EFFGAN is able to learn the data
distribution over all clients and mitigate client drift. It is able to train
with a large number of local epochs, making it more communication efficient
than previous works.
Related papers
- Towards Instance-adaptive Inference for Federated Learning [80.38701896056828]
Federated learning (FL) is a distributed learning paradigm that enables multiple clients to learn a powerful global model by aggregating local training.
In this paper, we present a novel FL algorithm, i.e., FedIns, to handle intra-client data heterogeneity by enabling instance-adaptive inference in the FL framework.
Our experiments show that our FedIns outperforms state-of-the-art FL algorithms, e.g., a 6.64% improvement against the top-performing method with less than 15% communication cost on Tiny-ImageNet.
arXiv Detail & Related papers (2023-08-11T09:58:47Z) - Efficient Distribution Similarity Identification in Clustered Federated
Learning via Principal Angles Between Client Data Subspaces [59.33965805898736]
Clustered learning has been shown to produce promising results by grouping clients into clusters.
Existing FL algorithms are essentially trying to group clients together with similar distributions.
Prior FL algorithms attempt similarities indirectly during training.
arXiv Detail & Related papers (2022-09-21T17:37:54Z) - FRAug: Tackling Federated Learning with Non-IID Features via
Representation Augmentation [31.12851987342467]
Federated Learning (FL) is a decentralized learning paradigm in which multiple clients collaboratively train deep learning models.
We propose Federated Representation Augmentation (FRAug) to tackle this practical and challenging problem.
Our approach generates synthetic client-specific samples in the embedding space to augment the usually small client datasets.
arXiv Detail & Related papers (2022-05-30T07:43:42Z) - FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling
and Correction [48.85303253333453]
Federated learning (FL) allows multiple clients to collectively train a high-performance global model without sharing their private data.
We propose a novel federated learning algorithm with local drift decoupling and correction (FedDC)
Our FedDC only introduces lightweight modifications in the local training phase, in which each client utilizes an auxiliary local drift variable to track the gap between the local model parameter and the global model parameters.
Experiment results and analysis demonstrate that FedDC yields expediting convergence and better performance on various image classification tasks.
arXiv Detail & Related papers (2022-03-22T14:06:26Z) - Data Selection for Efficient Model Update in Federated Learning [0.07614628596146598]
We propose to reduce the amount of local data that is needed to train a global model.
We do this by splitting the model into a lower part for generic feature extraction and an upper part that is more sensitive to the characteristics of the local data.
Our experiments show that less than 1% of the local data can transfer the characteristics of the client data to the global model.
arXiv Detail & Related papers (2021-11-05T14:07:06Z) - Federated Multi-Target Domain Adaptation [99.93375364579484]
Federated learning methods enable us to train machine learning models on distributed user data while preserving its privacy.
We consider a more practical scenario where the distributed client data is unlabeled, and a centralized labeled dataset is available on the server.
We propose an effective DualAdapt method to address the new challenges.
arXiv Detail & Related papers (2021-08-17T17:53:05Z) - Decentralized federated learning of deep neural networks on non-iid data [0.6335848702857039]
We tackle the non-problem of learning a personalized deep learning model in a decentralized setting.
We propose a method named Performance-Based Neighbor Selection (PENS) where clients with similar data detect each other and cooperate.
PENS is able to achieve higher accuracies as compared to strong baselines.
arXiv Detail & Related papers (2021-07-18T19:05:44Z) - Federated Learning with Taskonomy for Non-IID Data [0.0]
We introduce federated learning with taskonomy.
In a one-off process, the server provides the clients with a pretrained (and fine-tunable) encoder to compress their data into a latent representation, and transmit the signature of their data back to the server.
The server then learns the task-relatedness among clients via manifold learning, and performs a generalization of federated averaging.
arXiv Detail & Related papers (2021-03-29T20:47:45Z) - Exploiting Shared Representations for Personalized Federated Learning [54.65133770989836]
We propose a novel federated learning framework and algorithm for learning a shared data representation across clients and unique local heads for each client.
Our algorithm harnesses the distributed computational power across clients to perform many local-updates with respect to the low-dimensional local parameters for every update of the representation.
This result is of interest beyond federated learning to a broad class of problems in which we aim to learn a shared low-dimensional representation among data distributions.
arXiv Detail & Related papers (2021-02-14T05:36:25Z) - A Bayesian Federated Learning Framework with Online Laplace
Approximation [144.7345013348257]
Federated learning allows multiple clients to collaboratively learn a globally shared model.
We propose a novel FL framework that uses online Laplace approximation to approximate posteriors on both the client and server side.
We achieve state-of-the-art results on several benchmarks, clearly demonstrating the advantages of the proposed method.
arXiv Detail & Related papers (2021-02-03T08:36:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.