Providing Differential Privacy for Federated Learning Over Wireless: A Cross-layer Framework

• URL: http://arxiv.org/abs/2412.04408v1
• Date: Thu, 05 Dec 2024 18:27:09 GMT
• Title: Providing Differential Privacy for Federated Learning Over Wireless: A Cross-layer Framework
• Authors: Jiayu Mao, Tongxin Yin, Aylin Yener, Mingyan Liu,
• Abstract summary: Federated Learning (FL) is a distributed machine learning framework that inherently allows edge devices to maintain their local training data. We propose a wireless physical layer (PHY) design for OTA-FL which improves differential privacy (DP) through a decentralized, dynamic power control. This adaptation showcases the flexibility and effectiveness of our design across different learning algorithms while maintaining a strong emphasis on privacy.
• Score: 19.381425127772054
• License:
• Abstract: Federated Learning (FL) is a distributed machine learning framework that inherently allows edge devices to maintain their local training data, thus providing some level of privacy. However, FL's model updates still pose a risk of privacy leakage, which must be mitigated. Over-the-air FL (OTA-FL) is an adapted FL design for wireless edge networks that leverages the natural superposition property of the wireless medium. We propose a wireless physical layer (PHY) design for OTA-FL which improves differential privacy (DP) through a decentralized, dynamic power control that utilizes both inherent Gaussian noise in the wireless channel and a cooperative jammer (CJ) for additional artificial noise generation when higher privacy levels are required. Although primarily implemented within the Upcycled-FL framework, where a resource-efficient method with first-order approximations is used at every even iteration to decrease the required information from clients, our power control strategy is applicable to any FL framework, including FedAvg and FedProx as shown in the paper. This adaptation showcases the flexibility and effectiveness of our design across different learning algorithms while maintaining a strong emphasis on privacy. Our design removes the need for client-side artificial noise injection for DP, utilizing a cooperative jammer to enhance privacy without affecting transmission efficiency for higher privacy demands. Privacy analysis is provided using the Moments Accountant method. We perform a convergence analysis for non-convex objectives to tackle heterogeneous data distributions, highlighting the inherent trade-offs between privacy and accuracy. Numerical results show that our approach with various FL algorithms outperforms the state-of-the-art under the same DP conditions on the non-i.i.d. FEMNIST dataset, and highlight the cooperative jammer's effectiveness in ensuring strict privacy.

Related papers

• Federated Low-Rank Adaptation with Differential Privacy over Wireless Networks [24.667581521367357]
  Federated fine-tuning (FedFT) mitigates some privacy issues by facilitating collaborative model training without the need to share raw data. The risk of privacy eavesdropping attacks in FedFT remains a concern, particularly in sensitive areas such as healthcare and finance. We propose a split FedFT framework with differential privacy (DP) over wireless networks.
  arXiv  Detail & Related papers  (2024-11-12T14:01:08Z)
• Immersion and Invariance-based Coding for Privacy-Preserving Federated Learning [1.4226399196408985]
  Federated learning (FL) has emerged as a method to preserve privacy in collaborative distributed learning. We introduce a privacy-preserving FL framework that combines differential privacy and system immersion tools from control theory. We demonstrate that the proposed privacy-preserving scheme can be tailored to offer any desired level of differential privacy for both local and global model parameters.
  arXiv  Detail & Related papers  (2024-09-25T15:04:42Z)
• Privacy-preserving Federated Primal-dual Learning for Non-convex and Non-smooth Problems with Model Sparsification [51.04894019092156]
  Federated learning (FL) has been recognized as a rapidly growing area, where the model is trained over clients under the FL orchestration (PS) In this paper, we propose a novel primal sparification algorithm for and guarantee non-smooth FL problems. Its unique insightful properties and its analyses are also presented.
  arXiv  Detail & Related papers  (2023-10-30T14:15:47Z)
• Binary Federated Learning with Client-Level Differential Privacy [7.854806519515342]
  Federated learning (FL) is a privacy-preserving collaborative learning framework. Existing FL systems typically adopt Federated Average (FedAvg) as the training algorithm. We propose a communication-efficient FL training algorithm with differential privacy guarantee.
  arXiv  Detail & Related papers  (2023-08-07T06:07:04Z)
• Differentially Private Wireless Federated Learning Using Orthogonal Sequences [56.52483669820023]
  We propose a privacy-preserving uplink over-the-air computation (AirComp) method, termed FLORAS. We prove that FLORAS offers both item-level and client-level differential privacy guarantees. A new FL convergence bound is derived which, combined with the privacy guarantees, allows for a smooth tradeoff between the achieved convergence rate and differential privacy levels.
  arXiv  Detail & Related papers  (2023-06-14T06:35:10Z)
• Theoretically Principled Federated Learning for Balancing Privacy and Utility [61.03993520243198]
  We propose a general learning framework for the protection mechanisms that protects privacy via distorting model parameters. It can achieve personalized utility-privacy trade-off for each model parameter, on each client, at each communication round in federated learning.
  arXiv  Detail & Related papers  (2023-05-24T13:44:02Z)
• Over-the-Air Federated Averaging with Limited Power and Privacy Budgets [49.04036552090802]
  This paper studies a private over-the-air federated averaging (DP-OTA-FedAvg) system with a limited sum power budget. We aim to improve the analytical problem to minimize the gap of the DP-OTA-FedAvg coefficient to minimize privacy functions.
  arXiv  Detail & Related papers  (2023-05-05T13:56:40Z)
• Communication and Energy Efficient Wireless Federated Learning with Intrinsic Privacy [16.305837225117603]
  Federated Learning (FL) is a collaborative learning framework that enables edge devices to collaboratively learn a global model while keeping raw data locally. We propose a novel wireless FL scheme called private edge learning with spars (PFELS) to provide client-level DP guarantee with intrinsic channel noise.
  arXiv  Detail & Related papers  (2023-04-15T03:04:11Z)
• Over-the-Air Federated Learning with Privacy Protection via Correlated Additive Perturbations [57.20885629270732]
  We consider privacy aspects of wireless federated learning with Over-the-Air (OtA) transmission of gradient updates from multiple users/agents to an edge server. Traditional perturbation-based methods provide privacy protection while sacrificing the training accuracy. In this work, we aim at minimizing privacy leakage to the adversary and the degradation of model accuracy at the edge server.
  arXiv  Detail & Related papers  (2022-10-05T13:13:35Z)
• Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacy [67.4471689755097]
  This paper empirically demonstrates that the clipped FedAvg can perform surprisingly well even with substantial data heterogeneity. We provide the convergence analysis of a differential private (DP) FedAvg algorithm and highlight the relationship between clipping bias and the distribution of the clients' updates.
  arXiv  Detail & Related papers  (2021-06-25T14:47:19Z)
• Federated Learning with Sparsification-Amplified Privacy and Adaptive Optimization [27.243322019117144]
  Federated learning (FL) enables distributed agents to collaboratively learn a centralized model without sharing their raw data with each other. We propose a new FL framework with sparsification-amplified privacy. Our approach integrates random sparsification with gradient perturbation on each agent to amplify privacy guarantee.
  arXiv  Detail & Related papers  (2020-08-01T20:22:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.