FedLog: Personalized Federated Classification with Less Communication and More Flexibility

• URL: http://arxiv.org/abs/2407.08337v2
• Date: Sat, 12 Oct 2024 02:30:00 GMT
• Title: FedLog: Personalized Federated Classification with Less Communication and More Flexibility
• Authors: Haolin Yu, Guojun Zhang, Pascal Poupart,
• Abstract summary: Federated representation learning (FRL) aims to learn personalized federated models with effective feature extraction from local data. To reduce the overhead, we propose to share sufficient data summaries instead of raw model parameters.
• Score: 24.030147353437382
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Federated representation learning (FRL) aims to learn personalized federated models with effective feature extraction from local data. FRL algorithms that share the majority of the model parameters face significant challenges with huge communication overhead. This overhead stems from the millions of neural network parameters and slow aggregation progress of the averaging heuristic. To reduce the overhead, we propose to share sufficient data summaries instead of raw model parameters. The data summaries encode minimal sufficient statistics of an exponential family, and Bayesian inference is utilized for global aggregation. It helps to reduce message sizes and communication frequency. To further ensure formal privacy guarantee, we extend it with differential privacy framework. Empirical results demonstrate high learning accuracy with low communication overhead of our method.

Related papers

• Ferret: Federated Full-Parameter Tuning at Scale for Large Language Models [54.02863371927658]
  Large Language Models (LLMs) have become indispensable in numerous real-world applications. Ferret is the first first-order method with shared randomness. It achieves high computational efficiency, reduced communication overhead, and fast convergence.
  arXiv  Detail & Related papers  (2024-09-10T07:28:13Z)
• Personalized federated learning based on feature fusion [2.943623084019036]
  Federated learning enables distributed clients to collaborate on training while storing their data locally to protect client privacy. We propose a personalized federated learning approach called pFedPM. In our process, we replace traditional gradient uploading with feature uploading, which helps reduce communication costs and allows for heterogeneous client models.
  arXiv  Detail & Related papers  (2024-06-24T12:16:51Z)
• SpaFL: Communication-Efficient Federated Learning with Sparse Models and Low computational Overhead [75.87007729801304]
  SpaFL: a communication-efficient FL framework is proposed to optimize sparse model structures with low computational overhead. Experiments show that SpaFL improves accuracy while requiring much less communication and computing resources compared to sparse baselines.
  arXiv  Detail & Related papers  (2024-06-01T13:10:35Z)
• An Aggregation-Free Federated Learning for Tackling Data Heterogeneity [50.44021981013037]
  Federated Learning (FL) relies on the effectiveness of utilizing knowledge from distributed datasets. Traditional FL methods adopt an aggregate-then-adapt framework, where clients update local models based on a global model aggregated by the server from the previous training round. We introduce FedAF, a novel aggregation-free FL algorithm.
  arXiv  Detail & Related papers  (2024-04-29T05:55:23Z)
• Tunable Soft Prompts are Messengers in Federated Learning [55.924749085481544]
  Federated learning (FL) enables multiple participants to collaboratively train machine learning models using decentralized data sources. The lack of model privacy protection in FL becomes an unneglectable challenge. We propose a novel FL training approach that accomplishes information exchange among participants via tunable soft prompts.
  arXiv  Detail & Related papers  (2023-11-12T11:01:10Z)
• FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learning [87.08902493524556]
  Federated learning(FL) has recently attracted increasing attention from academia and industry. We propose FedDM to build the global training objective from multiple local surrogate functions. In detail, we construct synthetic sets of data on each client to locally match the loss landscape from original data.
  arXiv  Detail & Related papers  (2022-07-20T04:55:18Z)
• Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning [61.488646649045215]
  Federated learning (FL) is a promising strategy for performing privacy-preserving, distributed learning with a network of clients (i.e., edge devices)
  arXiv  Detail & Related papers  (2021-11-28T19:03:39Z)
• Personalized Federated Learning via Maximizing Correlation with Sparse and Hierarchical Extensions [14.862798952297105]
  Federated Learning (FL) is a collaborative machine learning technique to train a global model without obtaining clients' private data. We propose a novel personalized federated learning via maximizing correlation pFedMac. We show that pFedMac performs better than the L2-norm distance based personalization methods.
  arXiv  Detail & Related papers  (2021-07-12T11:43:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.