Communication Efficient and Privacy-Preserving Federated Learning Based on Evolution Strategies

• URL: http://arxiv.org/abs/2311.03405v2
• Date: Thu, 9 Nov 2023 01:41:36 GMT
• Title: Communication Efficient and Privacy-Preserving Federated Learning Based on Evolution Strategies
• Authors: Guangchen Lan
• Abstract summary: Federated learning (FL) is an emerging paradigm for training deep neural networks (DNNs) in distributed manners. In this work, we present a federated learning algorithm based on evolution strategies (FedES), a zeroth-order training method.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated learning (FL) is an emerging paradigm for training deep neural networks (DNNs) in distributed manners. Current FL approaches all suffer from high communication overhead and information leakage. In this work, we present a federated learning algorithm based on evolution strategies (FedES), a zeroth-order training method. Instead of transmitting model parameters, FedES only communicates loss values, and thus has very low communication overhead. Moreover, a third party is unable to estimate gradients without knowing the pre-shared seed, which protects data privacy. Experimental results demonstrate FedES can achieve the above benefits while keeping convergence performance the same as that with back propagation methods.

Related papers

• Gradient-Congruity Guided Federated Sparse Training [31.793271982853188]
  Federated learning (FL) is a distributed machine learning technique that facilitates this process while preserving data privacy. FL also faces challenges such as high computational and communication costs regarding resource-constrained devices. We propose the Gradient-Congruity Guided Federated Sparse Training (FedSGC), a novel method that integrates dynamic sparse training and gradient congruity inspection into federated learning framework.
  arXiv  Detail & Related papers  (2024-05-02T11:29:48Z)
• FedDBL: Communication and Data Efficient Federated Deep-Broad Learning for Histopathological Tissue Classification [65.7405397206767]
  We propose Federated Deep-Broad Learning (FedDBL) to achieve superior classification performance with limited training samples and only one-round communication. FedDBL greatly outperforms the competitors with only one-round communication and limited training samples, while it even achieves comparable performance with the ones under multiple-round communications. Since no data or deep model sharing across different clients, the privacy issue is well-solved and the model security is guaranteed with no model inversion attack risk.
  arXiv  Detail & Related papers  (2023-02-24T14:27:41Z)
• Magnitude Matters: Fixing SIGNSGD Through Magnitude-Aware Sparsification in the Presence of Data Heterogeneity [60.791736094073]
  Communication overhead has become one of the major bottlenecks in the distributed training of deep neural networks. We propose a magnitude-driven sparsification scheme, which addresses the non-convergence issue of SIGNSGD. The proposed scheme is validated through experiments on Fashion-MNIST, CIFAR-10, and CIFAR-100 datasets.
  arXiv  Detail & Related papers  (2023-02-19T17:42:35Z)
• Preserving Privacy in Federated Learning with Ensemble Cross-Domain Knowledge Distillation [22.151404603413752]
  Federated Learning (FL) is a machine learning paradigm where local nodes collaboratively train a central model. Existing FL methods typically share model parameters or employ co-distillation to address the issue of unbalanced data distribution. We develop a privacy preserving and communication efficient method in a FL framework with one-shot offline knowledge distillation.
  arXiv  Detail & Related papers  (2022-09-10T05:20:31Z)
• Acceleration of Federated Learning with Alleviated Forgetting in Local Training [61.231021417674235]
  Federated learning (FL) enables distributed optimization of machine learning models while protecting privacy. We propose FedReg, an algorithm to accelerate FL with alleviated knowledge forgetting in the local training stage. Our experiments demonstrate that FedReg not only significantly improves the convergence rate of FL, especially when the neural network architecture is deep.
  arXiv  Detail & Related papers  (2022-03-05T02:31:32Z)
• Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning [61.488646649045215]
  Federated learning (FL) is a promising strategy for performing privacy-preserving, distributed learning with a network of clients (i.e., edge devices)
  arXiv  Detail & Related papers  (2021-11-28T19:03:39Z)
• CosSGD: Nonlinear Quantization for Communication-efficient Federated Learning [62.65937719264881]
  Federated learning facilitates learning across clients without transferring local data on these clients to a central server. We propose a nonlinear quantization for compressed gradient descent, which can be easily utilized in federated learning. Our system significantly reduces the communication cost by up to three orders of magnitude, while maintaining convergence and accuracy of the training process.
  arXiv  Detail & Related papers  (2020-12-15T12:20:28Z)
• WAFFLe: Weight Anonymized Factorization for Federated Learning [88.44939168851721]
  In domains where data are sensitive or private, there is great value in methods that can learn in a distributed manner without the data ever leaving the local devices. We propose Weight Anonymized Factorization for Federated Learning (WAFFLe), an approach that combines the Indian Buffet Process with a shared dictionary of weight factors for neural networks.
  arXiv  Detail & Related papers  (2020-08-13T04:26:31Z)
• FedBoosting: Federated Learning with Gradient Protected Boosting for Text Recognition [7.988454173034258]
  Federated Learning (FL) framework allows learning a shared model collaboratively without data being centralized or shared among data owners. We show in this paper that the generalization ability of the joint model is poor on Non-Independent and Non-Identically Distributed (Non-IID) data. We propose a novel boosting algorithm for FL to address both the generalization and gradient leakage issues.
  arXiv  Detail & Related papers  (2020-07-14T18:47:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.