Accelerating Differentially Private Federated Learning via Adaptive Extrapolation

• URL: http://arxiv.org/abs/2504.09850v1
• Date: Mon, 14 Apr 2025 03:43:27 GMT
• Title: Accelerating Differentially Private Federated Learning via Adaptive Extrapolation
• Authors: Shokichi Takakura, Seng Pei Liew, Satoshi Hasegawa,
• Abstract summary: We propose DP-FedEXP, which adaptively selects the global step size based on the diversity of the local updates.<n>We show that DP-FedEXP provably accelerates the convergence of DP-FedAvg and it empirically outperforms existing methods tailored for DP-FL.
• Score: 2.6108066206600555
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The federated learning (FL) framework enables multiple clients to collaboratively train machine learning models without sharing their raw data, but it remains vulnerable to privacy attacks. One promising approach is to incorporate differential privacy (DP)-a formal notion of privacy-into the FL framework. DP-FedAvg is one of the most popular algorithms for DP-FL, but it is known to suffer from the slow convergence in the presence of heterogeneity among clients' data. Most of the existing methods to accelerate DP-FL require 1) additional hyperparameters or 2) additional computational cost for clients, which is not desirable since 1) hyperparameter tuning is computationally expensive and data-dependent choice of hyperparameters raises the risk of privacy leakage, and 2) clients are often resource-constrained. To address this issue, we propose DP-FedEXP, which adaptively selects the global step size based on the diversity of the local updates without requiring any additional hyperparameters or client computational cost. We show that DP-FedEXP provably accelerates the convergence of DP-FedAvg and it empirically outperforms existing methods tailored for DP-FL.

Related papers

• Client-Centric Federated Adaptive Optimization [78.30827455292827]
  Federated Learning (FL) is a distributed learning paradigm where clients collaboratively train a model while keeping their own data private.<n>We propose Federated-Centric Adaptive Optimization, which is a class of novel federated optimization approaches.
  arXiv  Detail & Related papers  (2025-01-17T04:00:50Z)
• Lightweight Federated Learning with Differential Privacy and Straggler Resilience [19.94124499453864]
  Federated learning (FL) enables collaborative model training through model parameter exchanges instead of raw data.<n>To avoid potential inference attacks from exchanged parameters, differential privacy (DP) offers rigorous guarantee against various attacks.<n>We propose LightDP-FL, a novel lightweight scheme that ensures provable DP against untrusted peers and server.
  arXiv  Detail & Related papers  (2024-12-09T00:54:00Z)
• Noise-Aware Algorithm for Heterogeneous Differentially Private Federated Learning [21.27813247914949]
  We propose Robust-HDP, which efficiently estimates the true noise level in clients model updates.<n>It improves utility and convergence speed, while being safe to the clients that may maliciously send falsified privacy parameter to server.
  arXiv  Detail & Related papers  (2024-06-05T17:41:42Z)
• PeFAD: A Parameter-Efficient Federated Framework for Time Series Anomaly Detection [51.20479454379662]
  We propose a. Federated Anomaly Detection framework named PeFAD with the increasing privacy concerns. We conduct extensive evaluations on four real datasets, where PeFAD outperforms existing state-of-the-art baselines by up to 28.74%.
  arXiv  Detail & Related papers  (2024-06-04T13:51:08Z)
• DP-DyLoRA: Fine-Tuning Transformer-Based Models On-Device under Differentially Private Federated Learning using Dynamic Low-Rank Adaptation [15.023077875990614]
  Federated learning (FL) allows clients to collaboratively train a global model without sharing their local data with a server. Differential privacy (DP) addresses such leakage by providing formal privacy guarantees, with mechanisms that add randomness to the clients' contributions. We propose an adaptation method that can be combined with differential privacy and call it DP-DyLoRA.
  arXiv  Detail & Related papers  (2024-05-10T10:10:37Z)
• DP-BREM: Differentially-Private and Byzantine-Robust Federated Learning with Client Momentum [11.68347496182345]
  Federated Learning (FL) allows multiple participating clients to train machine learning models collaboratively. Existing FL protocols are vulnerable to attacks that aim to compromise data privacy and/or model robustness. We focus on simultaneously achieving differential privacy (DP) and Byzantine robustness for cross-silo FL.
  arXiv  Detail & Related papers  (2023-06-22T00:11:53Z)
• Personalized Federated Learning under Mixture of Distributions [98.25444470990107]
  We propose a novel approach to Personalized Federated Learning (PFL), which utilizes Gaussian mixture models (GMM) to fit the input data distributions across diverse clients. FedGMM possesses an additional advantage of adapting to new clients with minimal overhead, and it also enables uncertainty quantification. Empirical evaluations on synthetic and benchmark datasets demonstrate the superior performance of our method in both PFL classification and novel sample detection.
  arXiv  Detail & Related papers  (2023-05-01T20:04:46Z)
• Differentially Private Federated Bayesian Optimization with Distributed Exploration [48.9049546219643]
  We introduce differential privacy (DP) into the training of deep neural networks through a general framework for adding DP to iterative algorithms. We show that DP-FTS-DE achieves high utility (competitive performance) with a strong privacy guarantee. We also use real-world experiments to show that DP-FTS-DE induces a trade-off between privacy and utility.
  arXiv  Detail & Related papers  (2021-10-27T04:11:06Z)
• Low-Latency Federated Learning over Wireless Channels with Differential Privacy [142.5983499872664]
  In federated learning (FL), model training is distributed over clients and local models are aggregated by a central server. In this paper, we aim to minimize FL training delay over wireless channels, constrained by overall training performance as well as each client's differential privacy (DP) requirement.
  arXiv  Detail & Related papers  (2021-06-20T13:51:18Z)
• On the Practicality of Differential Privacy in Federated Learning by Tuning Iteration Times [51.61278695776151]
  Federated Learning (FL) is well known for its privacy protection when training machine learning models among distributed clients collaboratively. Recent studies have pointed out that the naive FL is susceptible to gradient leakage attacks. Differential Privacy (DP) emerges as a promising countermeasure to defend against gradient leakage attacks.
  arXiv  Detail & Related papers  (2021-01-11T19:43:12Z)
• Federated Learning with Sparsification-Amplified Privacy and Adaptive Optimization [27.243322019117144]
  Federated learning (FL) enables distributed agents to collaboratively learn a centralized model without sharing their raw data with each other. We propose a new FL framework with sparsification-amplified privacy. Our approach integrates random sparsification with gradient perturbation on each agent to amplify privacy guarantee.
  arXiv  Detail & Related papers  (2020-08-01T20:22:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.