On Joint Noise Scaling in Differentially Private Federated Learning with Multiple Local Steps

• URL: http://arxiv.org/abs/2407.19286v1
• Date: Sat, 27 Jul 2024 15:54:58 GMT
• Title: On Joint Noise Scaling in Differentially Private Federated Learning with Multiple Local Steps
• Authors: Mikko A. Heikkilä,
• Abstract summary: Federated learning is a distributed learning setting where the main aim is to train machine learning models without having to share raw data. We show how a simple new analysis allows the parties to perform multiple local optimisation steps while still benefiting from secure aggregation.
• Score: 0.5439020425818999
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Federated learning is a distributed learning setting where the main aim is to train machine learning models without having to share raw data but only what is required for learning. To guarantee training data privacy and high-utility models, differential privacy and secure aggregation techniques are often combined with federated learning. However, with fine-grained protection granularities the currently existing techniques require the parties to communicate for each local optimisation step, if they want to fully benefit from the secure aggregation in terms of the resulting formal privacy guarantees. In this paper, we show how a simple new analysis allows the parties to perform multiple local optimisation steps while still benefiting from joint noise scaling when using secure aggregation. We show that our analysis enables higher utility models with guaranteed privacy protection under limited number of communication rounds.

Related papers

• Personalized Federated Learning for Cross-view Geo-localization [49.40531019551957]
  We propose a methodology combining Federated Learning (FL) with Cross-view Image Geo-localization (CVGL) techniques. Our method implements a coarse-to-fine approach, where clients share only the coarse feature extractors while keeping fine-grained features specific to local environments. Results demonstrate that our federated CVGL method achieves performance close to centralized training while maintaining data privacy.
  arXiv  Detail & Related papers  (2024-11-07T13:25:52Z)
• Federated Face Forgery Detection Learning with Personalized Representation [63.90408023506508]
  Deep generator technology can produce high-quality fake videos that are indistinguishable, posing a serious social threat. Traditional forgery detection methods directly centralized training on data. The paper proposes a novel federated face forgery detection learning with personalized representation.
  arXiv  Detail & Related papers  (2024-06-17T02:20:30Z)
• FewFedPIT: Towards Privacy-preserving and Few-shot Federated Instruction Tuning [54.26614091429253]
  Federated instruction tuning (FedIT) is a promising solution, by consolidating collaborative training across multiple data owners. FedIT encounters limitations such as scarcity of instructional data and risk of exposure to training data extraction attacks. We propose FewFedPIT, designed to simultaneously enhance privacy protection and model performance of federated few-shot learning.
  arXiv  Detail & Related papers  (2024-03-10T08:41:22Z)
• QMGeo: Differentially Private Federated Learning via Stochastic Quantization with Mixed Truncated Geometric Distribution [1.565361244756411]
  Federated learning (FL) is a framework which allows multiple users to jointly train a global machine learning (ML) model. One key motivation of such distributed frameworks is to provide privacy guarantees to the users. We present a novel quantization method, utilizing a mixed geometric distribution to introduce the randomness needed to provide DP.
  arXiv  Detail & Related papers  (2023-12-10T04:44:53Z)
• Balancing Privacy and Performance for Private Federated Learning Algorithms [4.681076651230371]
  Federated learning (FL) is a distributed machine learning framework where multiple clients collaborate to train a model without exposing their private data. FL algorithms frequently employ a differential privacy mechanism that introduces noise into each client's model updates before sharing. We show that an optimal balance exists between the number of local steps and communication rounds, one that maximizes the convergence performance within a given privacy budget.
  arXiv  Detail & Related papers  (2023-04-11T10:42:11Z)
• Certified Robustness in Federated Learning [54.03574895808258]
  We study the interplay between federated training, personalization, and certified robustness. We find that the simple federated averaging technique is effective in building not only more accurate, but also more certifiably-robust models.
  arXiv  Detail & Related papers  (2022-06-06T12:10:53Z)
• Personalization Improves Privacy-Accuracy Tradeoffs in Federated Optimization [57.98426940386627]
  We show that coordinating local learning with private centralized learning yields a generically useful and improved tradeoff between accuracy and privacy. We illustrate our theoretical results with experiments on synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2022-02-10T20:44:44Z)
• Efficient Differentially Private Secure Aggregation for Federated Learning via Hardness of Learning with Errors [1.4680035572775534]
  Federated machine learning leverages edge computing to develop models from network user data. Privacy in federated learning remains a major challenge. Recent advances in emphsecure aggregation using multiparty computation eliminate the need for a third party. We present a new federated learning protocol that leverages a novel differentially private, malicious secure aggregation protocol.
  arXiv  Detail & Related papers  (2021-12-13T18:31:08Z)
• Differentially private cross-silo federated learning [16.38610531397378]
  Strict privacy is of paramount importance in distributed machine learning. In this paper we combine additively homomorphic secure summation protocols with differential privacy in the so-called cross-silo federated learning setting. We demonstrate that our proposed solutions give prediction accuracy that is comparable to the non-distributed setting.
  arXiv  Detail & Related papers  (2020-07-10T18:15:10Z)
• Differentially Private Federated Learning with Laplacian Smoothing [72.85272874099644]
  Federated learning aims to protect data privacy by collaboratively learning a model without sharing private data among users. An adversary may still be able to infer the private training data by attacking the released model. Differential privacy provides a statistical protection against such attacks at the price of significantly degrading the accuracy or utility of the trained models.
  arXiv  Detail & Related papers  (2020-05-01T04:28:38Z)
• Concentrated Differentially Private and Utility Preserving Federated Learning [24.239992194656164]
  Federated learning is a machine learning setting where a set of edge devices collaboratively train a model under the orchestration of a central server. In this paper, we develop a federated learning approach that addresses the privacy challenge without much degradation on model utility. We provide a tight end-to-end privacy guarantee of our approach and analyze its theoretical convergence rates.
  arXiv  Detail & Related papers  (2020-03-30T19:20:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.