Behavior Mimics Distribution: Combining Individual and Group Behaviors for Federated Learning

• URL: http://arxiv.org/abs/2106.12300v1
• Date: Wed, 23 Jun 2021 10:42:37 GMT
• Title: Behavior Mimics Distribution: Combining Individual and Group Behaviors for Federated Learning
• Authors: Hua Huang, Fanhua Shang, Yuanyuan Liu, Hongying Liu
• Abstract summary: Federated Learning (FL) has become an active and promising distributed machine learning paradigm. Recent studies show that the performance of popular FL methods deteriorates dramatically due to the client drift caused by local updates. This paper proposes a novel Federated Learning algorithm (called IGFL), which leverages both Individual and Group behaviors to mimic distribution.
• Score: 26.36851197666568
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated Learning (FL) has become an active and promising distributed machine learning paradigm. As a result of statistical heterogeneity, recent studies clearly show that the performance of popular FL methods (e.g., FedAvg) deteriorates dramatically due to the client drift caused by local updates. This paper proposes a novel Federated Learning algorithm (called IGFL), which leverages both Individual and Group behaviors to mimic distribution, thereby improving the ability to deal with heterogeneity. Unlike existing FL methods, our IGFL can be applied to both client and server optimization. As a by-product, we propose a new attention-based federated learning in the server optimization of IGFL. To the best of our knowledge, this is the first time to incorporate attention mechanisms into federated optimization. We conduct extensive experiments and show that IGFL can significantly improve the performance of existing federated learning methods. Especially when the distributions of data among individuals are diverse, IGFL can improve the classification accuracy by about 13% compared with prior baselines.

Related papers

• Unlocking the Potential of Prompt-Tuning in Bridging Generalized and Personalized Federated Learning [49.72857433721424]
  Vision Transformers (ViT) and Visual Prompt Tuning (VPT) achieve state-of-the-art performance with improved efficiency in various computer vision tasks. We present a novel algorithm, SGPT, that integrates Generalized FL (GFL) and Personalized FL (PFL) approaches by employing a unique combination of both shared and group-specific prompts.
  arXiv  Detail & Related papers  (2023-10-27T17:22:09Z)
• PFL-GAN: When Client Heterogeneity Meets Generative Models in Personalized Federated Learning [55.930403371398114]
  We propose a novel generative adversarial network (GAN) sharing and aggregation strategy for personalized learning (PFL) PFL-GAN addresses the client heterogeneity in different scenarios. More specially, we first learn the similarity among clients and then develop an weighted collaborative data aggregation. The empirical results through the rigorous experimentation on several well-known datasets demonstrate the effectiveness of PFL-GAN.
  arXiv  Detail & Related papers  (2023-08-23T22:38:35Z)
• Stochastic Clustered Federated Learning [21.811496586350653]
  This paper proposes StoCFL, a novel clustered federated learning approach for generic Non-IID issues. In detail, StoCFL implements a flexible CFL framework that supports an arbitrary proportion of client participation and newly joined clients. The results show that StoCFL could obtain promising cluster results even when the number of clusters is unknown.
  arXiv  Detail & Related papers  (2023-03-02T01:39:16Z)
• FedRC: Tackling Diverse Distribution Shifts Challenge in Federated Learning by Robust Clustering [4.489171618387544]
  Federated Learning (FL) is a machine learning paradigm that safeguards privacy by retaining client data on edge devices. In this paper, we identify the learning challenges posed by the simultaneous occurrence of diverse distribution shifts. We propose a novel clustering algorithm framework, dubbed as FedRC, which adheres to our proposed clustering principle.
  arXiv  Detail & Related papers  (2023-01-29T06:50:45Z)
• FL Games: A Federated Learning Framework for Distribution Shifts [71.98708418753786]
  Federated learning aims to train predictive models for data that is distributed across clients, under the orchestration of a server. We propose FL GAMES, a game-theoretic framework for federated learning that learns causal features that are invariant across clients.
  arXiv  Detail & Related papers  (2022-10-31T22:59:03Z)
• Heterogeneous Federated Learning via Grouped Sequential-to-Parallel Training [60.892342868936865]
  Federated learning (FL) is a rapidly growing privacy-preserving collaborative machine learning paradigm. We propose a data heterogeneous-robust FL approach, FedGSP, to address this challenge. We show that FedGSP improves the accuracy by 3.7% on average compared with seven state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-01-31T03:15:28Z)
• On-the-fly Resource-Aware Model Aggregation for Federated Learning in Heterogeneous Edge [15.932747809197517]
  Edge computing has revolutionized the world of mobile and wireless networks world thanks to its flexible, secure, and performing characteristics. In this paper, we conduct an in-depth study of strategies to replace a central aggregation server with a flying master. Our results demonstrate a significant reduction of runtime using our flying master FL framework compared to the original FL from measurements results conducted in our EdgeAI testbed and over real 5G networks.
  arXiv  Detail & Related papers  (2021-12-21T19:04:42Z)
• Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning [61.488646649045215]
  Federated learning (FL) is a promising strategy for performing privacy-preserving, distributed learning with a network of clients (i.e., edge devices)
  arXiv  Detail & Related papers  (2021-11-28T19:03:39Z)
• Mobility-Aware Cluster Federated Learning in Hierarchical Wireless Networks [81.83990083088345]
  We develop a theoretical model to characterize the hierarchical federated learning (HFL) algorithm in wireless networks. Our analysis proves that the learning performance of HFL deteriorates drastically with highly-mobile users. To circumvent these issues, we propose a mobility-aware cluster federated learning (MACFL) algorithm.
  arXiv  Detail & Related papers  (2021-08-20T10:46:58Z)
• FedGroup: Efficient Clustered Federated Learning via Decomposed Data-Driven Measure [18.083188787905083]
  We propose a novel clustered federated learning (CFL) framework FedGroup. We show that FedGroup can significantly improve absolute test accuracy by +14.1% on FEMNIST compared to FedAvg. We also evaluate FedGroup and FedGrouProx (combined with FedProx) on several open datasets.
  arXiv  Detail & Related papers  (2020-10-14T08:15:34Z)
• TiFL: A Tier-based Federated Learning System [17.74678728280232]
  Federated Learning (FL) enables learning a shared model across many clients without violating the privacy requirements. We conduct a case study to show that heterogeneity in resource and data has a significant impact on training time and model accuracy in conventional FL systems. We propose TiFL, a Tier-based Federated Learning System, which divides clients into tiers based on their training performance and selects clients from the same tier in each training round.
  arXiv  Detail & Related papers  (2020-01-25T01:40:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.