Immersion and Invariance-based Coding for Privacy-Preserving Federated Learning

• URL: http://arxiv.org/abs/2409.17201v2
• Date: Mon, 25 Nov 2024 09:32:53 GMT
• Title: Immersion and Invariance-based Coding for Privacy-Preserving Federated Learning
• Authors: Haleh Hayati, Carlos Murguia, Nathan van de Wouw,
• Abstract summary: Federated learning (FL) has emerged as a method to preserve privacy in collaborative distributed learning. We introduce a privacy-preserving FL framework that combines differential privacy and system immersion tools from control theory. We demonstrate that the proposed privacy-preserving scheme can be tailored to offer any desired level of differential privacy for both local and global model parameters.
• Score: 1.4226399196408985
• License:
• Abstract: Federated learning (FL) has emerged as a method to preserve privacy in collaborative distributed learning. In FL, clients train AI models directly on their devices rather than sharing data with a centralized server, which can pose privacy risks. However, it has been shown that despite FL's partial protection of local data privacy, information about clients' data can still be inferred from shared model updates during training. In recent years, several privacy-preserving approaches have been developed to mitigate this privacy leakage in FL, though they often provide privacy at the cost of model performance or system efficiency. Balancing these trade-offs presents a significant challenge in implementing FL schemes. In this manuscript, we introduce a privacy-preserving FL framework that combines differential privacy and system immersion tools from control theory. The core idea is to treat the optimization algorithms used in standard FL schemes (e.g., gradient-based algorithms) as a dynamical system that we seek to immerse into a higher-dimensional system (referred to as the target optimization algorithm). The target algorithm's dynamics are designed such that, first, the model parameters of the original algorithm are immersed in its parameters; second, it operates on distorted parameters; and third, it converges to an encoded version of the true model parameters from the original algorithm. These encoded parameters can then be decoded at the server to retrieve the original model parameters. We demonstrate that the proposed privacy-preserving scheme can be tailored to offer any desired level of differential privacy for both local and global model parameters, while maintaining the same accuracy and convergence rate as standard FL algorithms.

Related papers

• CorBin-FL: A Differentially Private Federated Learning Mechanism using Common Randomness [6.881974834597426]
  Federated learning (FL) has emerged as a promising framework for distributed machine learning. We introduce CorBin-FL, a privacy mechanism that uses correlated binary quantization to achieve differential privacy. We also propose AugCorBin-FL, an extension that, in addition to PLDP, user-level and sample-level central differential privacy guarantees.
  arXiv  Detail & Related papers  (2024-09-20T00:23:44Z)
• Bridging Data Barriers among Participants: Assessing the Potential of Geoenergy through Federated Learning [2.8498944632323755]
  This study introduces a novel federated learning (FL) framework based on XGBoost models. FL models demonstrate superior accuracy and generalization capabilities compared to separate models. This study opens new avenues for assessing unconventional reservoirs through collaborative and privacy-preserving FL techniques.
  arXiv  Detail & Related papers  (2024-04-29T09:12:31Z)
• Privacy-preserving Federated Primal-dual Learning for Non-convex and Non-smooth Problems with Model Sparsification [51.04894019092156]
  Federated learning (FL) has been recognized as a rapidly growing area, where the model is trained over clients under the FL orchestration (PS) In this paper, we propose a novel primal sparification algorithm for and guarantee non-smooth FL problems. Its unique insightful properties and its analyses are also presented.
  arXiv  Detail & Related papers  (2023-10-30T14:15:47Z)
• Binary Federated Learning with Client-Level Differential Privacy [7.854806519515342]
  Federated learning (FL) is a privacy-preserving collaborative learning framework. Existing FL systems typically adopt Federated Average (FedAvg) as the training algorithm. We propose a communication-efficient FL training algorithm with differential privacy guarantee.
  arXiv  Detail & Related papers  (2023-08-07T06:07:04Z)
• Theoretically Principled Federated Learning for Balancing Privacy and Utility [61.03993520243198]
  We propose a general learning framework for the protection mechanisms that protects privacy via distorting model parameters. It can achieve personalized utility-privacy trade-off for each model parameter, on each client, at each communication round in federated learning.
  arXiv  Detail & Related papers  (2023-05-24T13:44:02Z)
• FedLAP-DP: Federated Learning by Sharing Differentially Private Loss Approximations [53.268801169075836]
  We propose FedLAP-DP, a novel privacy-preserving approach for federated learning. A formal privacy analysis demonstrates that FedLAP-DP incurs the same privacy costs as typical gradient-sharing schemes. Our approach presents a faster convergence speed compared to typical gradient-sharing methods.
  arXiv  Detail & Related papers  (2023-02-02T12:56:46Z)
• sqSGD: Locally Private and Communication Efficient Federated Learning [14.60645909629309]
  Federated learning (FL) is a technique that trains machine learning models from decentralized data sources. We develop a gradient-based learning algorithm called sqSGD that addresses communication efficiency and high-dimensional compatibility. Experiment results show sqSGD successfully learns large models like LeNet and ResNet with local privacy constraints.
  arXiv  Detail & Related papers  (2022-06-21T17:45:35Z)
• Privacy-Preserving Federated Learning via System Immersion and Random Matrix Encryption [4.258856853258348]
  Federated learning (FL) has emerged as a privacy solution for collaborative distributed learning where clients train AI models directly on their devices instead of sharing their data with a centralized (potentially adversarial) server. We propose a Privacy-Preserving Federated Learning (PPFL) framework built on the synergy of matrix encryption and system immersion tools from control theory. We show that our algorithm provides the same level of accuracy and convergence rate as the standard FL with a negligible cost while revealing no information about clients' data.
  arXiv  Detail & Related papers  (2022-04-05T21:28:59Z)
• Acceleration of Federated Learning with Alleviated Forgetting in Local Training [61.231021417674235]
  Federated learning (FL) enables distributed optimization of machine learning models while protecting privacy. We propose FedReg, an algorithm to accelerate FL with alleviated knowledge forgetting in the local training stage. Our experiments demonstrate that FedReg not only significantly improves the convergence rate of FL, especially when the neural network architecture is deep.
  arXiv  Detail & Related papers  (2022-03-05T02:31:32Z)
• Do Gradient Inversion Attacks Make Federated Learning Unsafe? [70.0231254112197]
  Federated learning (FL) allows the collaborative training of AI models without needing to share raw data. Recent works on the inversion of deep neural networks from model gradients raised concerns about the security of FL in preventing the leakage of training data. In this work, we show that these attacks presented in the literature are impractical in real FL use-cases and provide a new baseline attack.
  arXiv  Detail & Related papers  (2022-02-14T18:33:12Z)
• Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacy [67.4471689755097]
  This paper empirically demonstrates that the clipped FedAvg can perform surprisingly well even with substantial data heterogeneity. We provide the convergence analysis of a differential private (DP) FedAvg algorithm and highlight the relationship between clipping bias and the distribution of the clients' updates.
  arXiv  Detail & Related papers  (2021-06-25T14:47:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.