Client Adaptation improves Federated Learning with Simulated Non-IID Clients

• URL: http://arxiv.org/abs/2007.04806v1
• Date: Thu, 9 Jul 2020 13:48:39 GMT
• Title: Client Adaptation improves Federated Learning with Simulated Non-IID Clients
• Authors: Laura Rieger, Rasmus M. Th. H{\o}egh, and Lars K. Hansen
• Abstract summary: We present a federated learning approach for learning a client adaptable, robust model when data is non-identically and non-independently distributed (non-IID) across clients. We show that adding learned client-specific conditioning improves model performance, and the approach is shown to work on balanced and imbalanced data set from both audio and image domains.
• Score: 1.0896567381206714
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a federated learning approach for learning a client adaptable, robust model when data is non-identically and non-independently distributed (non-IID) across clients. By simulating heterogeneous clients, we show that adding learned client-specific conditioning improves model performance, and the approach is shown to work on balanced and imbalanced data set from both audio and image domains. The client adaptation is implemented by a conditional gated activation unit and is particularly beneficial when there are large differences between the data distribution for each client, a common scenario in federated learning.

Related papers

• Fine-Tuning Personalization in Federated Learning to Mitigate Adversarial Clients [8.773068878015856]
  Federated learning (FL) is an appealing paradigm that allows a group of machines (a.k.a. clients) to learn collectively while keeping their data local. We consider an FL setting where some clients can be adversarial, and we derive conditions under which full collaboration fails.
  arXiv  Detail & Related papers  (2024-09-30T14:31:19Z)
• Towards Instance-adaptive Inference for Federated Learning [80.38701896056828]
  Federated learning (FL) is a distributed learning paradigm that enables multiple clients to learn a powerful global model by aggregating local training. In this paper, we present a novel FL algorithm, i.e., FedIns, to handle intra-client data heterogeneity by enabling instance-adaptive inference in the FL framework. Our experiments show that our FedIns outperforms state-of-the-art FL algorithms, e.g., a 6.64% improvement against the top-performing method with less than 15% communication cost on Tiny-ImageNet.
  arXiv  Detail & Related papers  (2023-08-11T09:58:47Z)
• Adaptive Federated Learning with Auto-Tuned Clients [8.868957280690832]
  Federated learning (FL) is a distributed machine learning framework where the global model of a central server is trained via multiple collaborative steps by participating clients without sharing their data. We propose $Delta$-SGD, a simple step size rule for SGD that enables each client to use its own step size by adapting to the local smoothness of the function each client is optimizing.
  arXiv  Detail & Related papers  (2023-06-19T23:46:42Z)
• FilFL: Client Filtering for Optimized Client Participation in Federated Learning [71.46173076298957]
  Federated learning enables clients to collaboratively train a model without exchanging local data. Clients participating in the training process significantly impact the convergence rate, learning efficiency, and model generalization. We propose a novel approach, client filtering, to improve model generalization and optimize client participation and training.
  arXiv  Detail & Related papers  (2023-02-13T18:55:31Z)
• When to Trust Aggregated Gradients: Addressing Negative Client Sampling in Federated Learning [41.51682329500003]
  We propose a novel learning rate adaptation mechanism to adjust the server learning rate for the aggregated gradient in each round. We make theoretical deductions to find a meaningful and robust indicator that is positively related to the optimal server learning rate.
  arXiv  Detail & Related papers  (2023-01-25T03:52:45Z)
• Straggler-Resilient Personalized Federated Learning [55.54344312542944]
  Federated learning allows training models from samples distributed across a large network of clients while respecting privacy and communication restrictions. We develop a novel algorithmic procedure with theoretical speedup guarantees that simultaneously handles two of these hurdles. Our method relies on ideas from representation learning theory to find a global common representation using all clients' data and learn a user-specific set of parameters leading to a personalized solution for each client.
  arXiv  Detail & Related papers  (2022-06-05T01:14:46Z)
• FRAug: Tackling Federated Learning with Non-IID Features via Representation Augmentation [31.12851987342467]
  Federated Learning (FL) is a decentralized learning paradigm in which multiple clients collaboratively train deep learning models. We propose Federated Representation Augmentation (FRAug) to tackle this practical and challenging problem. Our approach generates synthetic client-specific samples in the embedding space to augment the usually small client datasets.
  arXiv  Detail & Related papers  (2022-05-30T07:43:42Z)
• PerFED-GAN: Personalized Federated Learning via Generative Adversarial Networks [46.17495529441229]
  Federated learning is a distributed machine learning method that can be used to deploy AI-dependent IoT applications. This paper proposes a federated learning method based on co-training and generative adversarial networks(GANs) In our experiments, the proposed method outperforms the existing methods in mean test accuracy by 42% when the client's model architecture and data distribution vary significantly.
  arXiv  Detail & Related papers  (2022-02-18T12:08:46Z)
• Federated Multi-Target Domain Adaptation [99.93375364579484]
  Federated learning methods enable us to train machine learning models on distributed user data while preserving its privacy. We consider a more practical scenario where the distributed client data is unlabeled, and a centralized labeled dataset is available on the server. We propose an effective DualAdapt method to address the new challenges.
  arXiv  Detail & Related papers  (2021-08-17T17:53:05Z)
• Towards Fair Federated Learning with Zero-Shot Data Augmentation [123.37082242750866]
  Federated learning has emerged as an important distributed learning paradigm, where a server aggregates a global model from many client-trained models while having no access to the client data. We propose a novel federated learning system that employs zero-shot data augmentation on under-represented data to mitigate statistical heterogeneity and encourage more uniform accuracy performance across clients in federated networks. We study two variants of this scheme, Fed-ZDAC (federated learning with zero-shot data augmentation at the clients) and Fed-ZDAS (federated learning with zero-shot data augmentation at the server).
  arXiv  Detail & Related papers  (2021-04-27T18:23:54Z)
• Toward Understanding the Influence of Individual Clients in Federated Learning [52.07734799278535]
  Federated learning allows clients to jointly train a global model without sending their private data to a central server. We defined a new notion called em-Influence, quantify this influence over parameters, and proposed an effective efficient model to estimate this metric.
  arXiv  Detail & Related papers  (2020-12-20T14:34:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.