Differentially-Private Multi-Tier Federated Learning

• URL: http://arxiv.org/abs/2401.11592v5
• Date: Fri, 08 Nov 2024 03:37:03 GMT
• Title: Differentially-Private Multi-Tier Federated Learning
• Authors: Evan Chen, Frank Po-Chen Lin, Dong-Jun Han, Christopher G. Brinton,
• Abstract summary: We propose Multi-Tier Federated Learning with Multi-Tier Differential Privacy (M2FDP) M2FDP is a DP-enhanced FL methodology for jointly optimizing privacy and performance in hierarchical networks.
• Score: 14.725823723623213
• License:
• Abstract: While federated learning (FL) eliminates the transmission of raw data over a network, it is still vulnerable to privacy breaches from the communicated model parameters. In this work, we propose Multi-Tier Federated Learning with Multi-Tier Differential Privacy (M^2FDP), a DP-enhanced FL methodology for jointly optimizing privacy and performance in hierarchical networks. One of the key concepts of M^2FDP is to extend the concept of HDP towards Multi-Tier Differential Privacy (MDP), while also adapting DP noise injection at different layers of an established FL hierarchy -- edge devices, edge servers, and cloud servers -- according to the trust models within particular subnetworks. We conduct a comprehensive analysis of the convergence behavior of M^2FDP, revealing conditions on parameter tuning under which the training process converges sublinearly to a finite stationarity gap that depends on the network hierarchy, trust model, and target privacy level. Subsequent numerical evaluations demonstrate that M^2FDP obtains substantial improvements in these metrics over baselines for different privacy budgets, and validate the impact of different system configurations.

Related papers

• DMM: Distributed Matrix Mechanism for Differentially-Private Federated Learning using Packed Secret Sharing [51.336015600778396]
  Federated Learning (FL) has gained lots of traction recently, both in industry and academia. In FL, a machine learning model is trained using data from various end-users arranged in committees across several rounds. Since such data can often be sensitive, a primary challenge in FL is providing privacy while still retaining utility of the model.
  arXiv  Detail & Related papers  (2024-10-21T16:25:14Z)
• DP$^2$-FedSAM: Enhancing Differentially Private Federated Learning Through Personalized Sharpness-Aware Minimization [8.022417295372492]
  Federated learning (FL) is a distributed machine learning approach that allows multiple clients to collaboratively train a model without sharing their raw data. To prevent sensitive information from being inferred through the model updates shared in FL, differentially private federated learning (DPFL) has been proposed. DPFL ensures formal and rigorous privacy protection in FL by clipping and adding random noise to the shared model updates. We propose DP$2$-FedSAM: Differentially Private and Personalized Federated Learning with Sharpness-Aware Minimization.
  arXiv  Detail & Related papers  (2024-09-20T16:49:01Z)
• Sequential Federated Learning in Hierarchical Architecture on Non-IID Datasets [25.010661914466354]
  In a real federated learning (FL) system, communication overhead for passing model parameters between the clients and the parameter (PS) is often a bottleneck. We propose sequential FL (SFL) HFL for the first time, which removes the central PS and enables the model to be completed only through passing data between two adjacent ESs for each server.
  arXiv  Detail & Related papers  (2024-08-19T07:43:35Z)
• Universally Harmonizing Differential Privacy Mechanisms for Federated Learning: Boosting Accuracy and Convergence [22.946928984205588]
  Differentially private federated learning (DP-FL) is a promising technique for collaborative model training. We propose the first DP-FL framework (namely UDP-FL) which universally harmonizes any randomization mechanism. We show that UDP-FL exhibits substantial resilience against different inference attacks.
  arXiv  Detail & Related papers  (2024-07-20T00:11:59Z)
• Pareto Low-Rank Adapters: Efficient Multi-Task Learning with Preferences [49.14535254003683]
  PaLoRA is a novel parameter-efficient method that augments the original model with task-specific low-rank adapters. Our experimental results show that PaLoRA outperforms MTL and PFL baselines across various datasets.
  arXiv  Detail & Related papers  (2024-07-10T21:25:51Z)
• Differentially Private Over-the-Air Federated Learning Over MIMO Fading Channels [24.534729104570417]
  Federated learning (FL) enables edge devices to collaboratively train machine learning models. While over-the-air model aggregation improves communication efficiency, uploading models to an edge server over wireless networks can pose privacy risks. We show that FL model communication with a multiple-antenna server amplifies privacy leakage.
  arXiv  Detail & Related papers  (2023-06-19T14:44:34Z)
• Differentially Private Wireless Federated Learning Using Orthogonal Sequences [56.52483669820023]
  We propose a privacy-preserving uplink over-the-air computation (AirComp) method, termed FLORAS. We prove that FLORAS offers both item-level and client-level differential privacy guarantees. A new FL convergence bound is derived which, combined with the privacy guarantees, allows for a smooth tradeoff between the achieved convergence rate and differential privacy levels.
  arXiv  Detail & Related papers  (2023-06-14T06:35:10Z)
• Amplitude-Varying Perturbation for Balancing Privacy and Utility in Federated Learning [86.08285033925597]
  This paper presents a new DP perturbation mechanism with a time-varying noise amplitude to protect the privacy of federated learning. We derive an online refinement of the series to prevent FL from premature convergence resulting from excessive perturbation noise. The contribution of the new DP mechanism to the convergence and accuracy of privacy-preserving FL is corroborated, compared to the state-of-the-art Gaussian noise mechanism with a persistent noise amplitude.
  arXiv  Detail & Related papers  (2023-03-07T22:52:40Z)
• FedLAP-DP: Federated Learning by Sharing Differentially Private Loss Approximations [53.268801169075836]
  We propose FedLAP-DP, a novel privacy-preserving approach for federated learning. A formal privacy analysis demonstrates that FedLAP-DP incurs the same privacy costs as typical gradient-sharing schemes. Our approach presents a faster convergence speed compared to typical gradient-sharing methods.
  arXiv  Detail & Related papers  (2023-02-02T12:56:46Z)
• Low-Latency Federated Learning over Wireless Channels with Differential Privacy [142.5983499872664]
  In federated learning (FL), model training is distributed over clients and local models are aggregated by a central server. In this paper, we aim to minimize FL training delay over wireless channels, constrained by overall training performance as well as each client's differential privacy (DP) requirement.
  arXiv  Detail & Related papers  (2021-06-20T13:51:18Z)
• LDP-Fed: Federated Learning with Local Differential Privacy [14.723892247530234]
  We present LDP-Fed, a novel federated learning system with a formal privacy guarantee using local differential privacy (LDP) Existing LDP protocols are developed primarily to ensure data privacy in the collection of single numerical or categorical values. In federated learning model parameter updates are collected iteratively from each participant.
  arXiv  Detail & Related papers  (2020-06-05T19:15:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.