Privacy-Preserving Federated Learning via System Immersion and Random Matrix Encryption

• URL: http://arxiv.org/abs/2204.02497v1
• Date: Tue, 5 Apr 2022 21:28:59 GMT
• Title: Privacy-Preserving Federated Learning via System Immersion and Random Matrix Encryption
• Authors: Haleh Hayati, Carlos Murguia, Nathan van de Wouw
• Abstract summary: Federated learning (FL) has emerged as a privacy solution for collaborative distributed learning where clients train AI models directly on their devices instead of sharing their data with a centralized (potentially adversarial) server. We propose a Privacy-Preserving Federated Learning (PPFL) framework built on the synergy of matrix encryption and system immersion tools from control theory. We show that our algorithm provides the same level of accuracy and convergence rate as the standard FL with a negligible cost while revealing no information about clients' data.
• Score: 4.258856853258348
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated learning (FL) has emerged as a privacy solution for collaborative distributed learning where clients train AI models directly on their devices instead of sharing their data with a centralized (potentially adversarial) server. Although FL preserves local data privacy to some extent, it has been shown that information about clients' data can still be inferred from model updates. In recent years, various privacy-preserving schemes have been developed to address this privacy leakage. However, they often provide privacy at the expense of model performance or system efficiency and balancing these tradeoffs is a crucial challenge when implementing FL schemes. In this manuscript, we propose a Privacy-Preserving Federated Learning (PPFL) framework built on the synergy of matrix encryption and system immersion tools from control theory. The idea is to immerse the learning algorithm, a Stochastic Gradient Decent (SGD), into a higher-dimensional system (the so-called target system) and design the dynamics of the target system so that: the trajectories of the original SGD are immersed/embedded in its trajectories, and it learns on encrypted data (here we use random matrix encryption). Matrix encryption is reformulated at the server as a random change of coordinates that maps original parameters to a higher-dimensional parameter space and enforces that the target SGD converges to an encrypted version of the original SGD optimal solution. The server decrypts the aggregated model using the left inverse of the immersion map. We show that our algorithm provides the same level of accuracy and convergence rate as the standard FL with a negligible computation cost while revealing no information about the clients' data.

Related papers

• DMM: Distributed Matrix Mechanism for Differentially-Private Federated Learning using Packed Secret Sharing [51.336015600778396]
  Federated Learning (FL) has gained lots of traction recently, both in industry and academia. In FL, a machine learning model is trained using data from various end-users arranged in committees across several rounds. Since such data can often be sensitive, a primary challenge in FL is providing privacy while still retaining utility of the model.
  arXiv  Detail & Related papers  (2024-10-21T16:25:14Z)
• Immersion and Invariance-based Coding for Privacy-Preserving Federated Learning [1.4226399196408985]
  Federated learning (FL) has emerged as a method to preserve privacy in collaborative distributed learning. We introduce a privacy-preserving FL framework that combines differential privacy and system immersion tools from control theory. We demonstrate that the proposed privacy-preserving scheme can be tailored to offer any desired level of differential privacy for both local and global model parameters.
  arXiv  Detail & Related papers  (2024-09-25T15:04:42Z)
• FLUE: Federated Learning with Un-Encrypted model weights [0.0]
  Federated learning enables devices to collaboratively train a shared model while keeping training data locally stored. Recent research emphasizes using encrypted model parameters during training. This paper introduces a novel federated learning algorithm, leveraging coded local gradients without encryption.
  arXiv  Detail & Related papers  (2024-07-26T14:04:57Z)
• Initialization Matters: Privacy-Utility Analysis of Overparameterized Neural Networks [72.51255282371805]
  We prove a privacy bound for the KL divergence between model distributions on worst-case neighboring datasets. We find that this KL privacy bound is largely determined by the expected squared gradient norm relative to model parameters during training.
  arXiv  Detail & Related papers  (2023-10-31T16:13:22Z)
• FheFL: Fully Homomorphic Encryption Friendly Privacy-Preserving Federated Learning with Byzantine Users [19.209830150036254]
  federated learning (FL) technique was developed to mitigate data privacy issues in the traditional machine learning paradigm. Next-generation FL architectures proposed encryption and anonymization techniques to protect the model updates from the server. This paper proposes a novel FL algorithm based on a fully homomorphic encryption (FHE) scheme.
  arXiv  Detail & Related papers  (2023-06-08T11:20:00Z)
• PEOPL: Characterizing Privately Encoded Open Datasets with Public Labels [59.66777287810985]
  We introduce information-theoretic scores for privacy and utility, which quantify the average performance of an unfaithful user. We then theoretically characterize primitives in building families of encoding schemes that motivate the use of random deep neural networks.
  arXiv  Detail & Related papers  (2023-03-31T18:03:53Z)
• sqSGD: Locally Private and Communication Efficient Federated Learning [14.60645909629309]
  Federated learning (FL) is a technique that trains machine learning models from decentralized data sources. We develop a gradient-based learning algorithm called sqSGD that addresses communication efficiency and high-dimensional compatibility. Experiment results show sqSGD successfully learns large models like LeNet and ResNet with local privacy constraints.
  arXiv  Detail & Related papers  (2022-06-21T17:45:35Z)
• Acceleration of Federated Learning with Alleviated Forgetting in Local Training [61.231021417674235]
  Federated learning (FL) enables distributed optimization of machine learning models while protecting privacy. We propose FedReg, an algorithm to accelerate FL with alleviated knowledge forgetting in the local training stage. Our experiments demonstrate that FedReg not only significantly improves the convergence rate of FL, especially when the neural network architecture is deep.
  arXiv  Detail & Related papers  (2022-03-05T02:31:32Z)
• Secure Neuroimaging Analysis using Federated Learning with Homomorphic Encryption [14.269757725951882]
  Federated learning (FL) enables distributed computation of machine learning models over disparate, remote data sources. Recent membership attacks show that private or sensitive personal data can sometimes be leaked or inferred when model parameters or summary statistics are shared with a central site. We propose a framework for secure FL using fully-homomorphic encryption (FHE)
  arXiv  Detail & Related papers  (2021-08-07T12:15:52Z)
• Differentially Private Federated Learning with Laplacian Smoothing [72.85272874099644]
  Federated learning aims to protect data privacy by collaboratively learning a model without sharing private data among users. An adversary may still be able to infer the private training data by attacking the released model. Differential privacy provides a statistical protection against such attacks at the price of significantly degrading the accuracy or utility of the trained models.
  arXiv  Detail & Related papers  (2020-05-01T04:28:38Z)
• Privacy-preserving Traffic Flow Prediction: A Federated Learning Approach [61.64006416975458]
  We propose a privacy-preserving machine learning technique named Federated Learning-based Gated Recurrent Unit neural network algorithm (FedGRU) for traffic flow prediction. FedGRU differs from current centralized learning methods and updates universal learning models through a secure parameter aggregation mechanism. It is shown that FedGRU's prediction accuracy is 90.96% higher than the advanced deep learning models.
  arXiv  Detail & Related papers  (2020-03-19T13:07:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.