Hybrid Differentially Private Federated Learning on Vertically Partitioned Data

• URL: http://arxiv.org/abs/2009.02763v1
• Date: Sun, 6 Sep 2020 16:06:04 GMT
• Title: Hybrid Differentially Private Federated Learning on Vertically Partitioned Data
• Authors: Chang Wang, Jian Liang, Mingkai Huang, Bing Bai, Kun Bai, Hao Li
• Abstract summary: We present HDP-VFL, the first hybrid differentially private (DP) framework for vertical federated learning (VFL) We analyze how VFL's intermediate result (IR) can leak private information of the training data during communication. We mathematically prove that our algorithm not only provides utility guarantees for VFL, but also offers multi-level privacy.
• Score: 41.7896466307821
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present HDP-VFL, the first hybrid differentially private (DP) framework for vertical federated learning (VFL) to demonstrate that it is possible to jointly learn a generalized linear model (GLM) from vertically partitioned data with only a negligible cost, w.r.t. training time, accuracy, etc., comparing to idealized non-private VFL. Our work builds on the recent advances in VFL-based collaborative training among different organizations which rely on protocols like Homomorphic Encryption (HE) and Secure Multi-Party Computation (MPC) to secure computation and training. In particular, we analyze how VFL's intermediate result (IR) can leak private information of the training data during communication and design a DP-based privacy-preserving algorithm to ensure the data confidentiality of VFL participants. We mathematically prove that our algorithm not only provides utility guarantees for VFL, but also offers multi-level privacy, i.e. DP w.r.t. IR and joint differential privacy (JDP) w.r.t. model weights. Experimental results demonstrate that our work, under adequate privacy budgets, is quantitatively and qualitatively similar to GLMs, learned in idealized non-private VFL setting, rather than the increased cost in memory and processing time in most prior works based on HE or MPC. Our codes will be released if this paper is accepted.

Related papers

• ALI-DPFL: Differentially Private Federated Learning with Adaptive Local Iterations [26.310416723272184]
  Federated Learning (FL) is a distributed machine learning technique that allows model training among multiple devices or organizations by sharing training parameters instead of raw data. adversaries can still infer individual information through inference attacks on these training parameters. Differential Privacy (DP) has been widely used in FL to prevent such attacks. We consider differentially private federated learning in a resource-constrained scenario, where both privacy budget and communication rounds are constrained.
  arXiv  Detail & Related papers  (2023-08-21T04:09:59Z)
• Can Public Large Language Models Help Private Cross-device Federated Learning? [58.05449579773249]
  We study (differentially) private federated learning (FL) of language models. Public data has been used to improve privacy-utility trade-offs for both large and small language models. We propose a novel distribution matching algorithm with theoretical grounding to sample public data close to private data distribution.
  arXiv  Detail & Related papers  (2023-05-20T07:55:58Z)
• Quadratic Functional Encryption for Secure Training in Vertical Federated Learning [26.188083606166806]
  Vertical federated learning (VFL) enables the collaborative training of machine learning (ML) models in settings where the data is distributed amongst multiple parties. In VFL, the labels are available to a single party and the complete feature set is formed only when data from all parties is combined. Recently, Xu et al. proposed a new framework called FedV for secure gradient computation for VFL using multi-input functional encryption.
  arXiv  Detail & Related papers  (2023-05-15T05:31:35Z)
• BlindFL: Vertical Federated Machine Learning without Peeking into Your Data [20.048695060411774]
  Vertical federated learning (VFL) describes a case where ML models are built upon the private data of different participated parties. We introduce BlindFL, a novel framework for VFL training and inference. We show that BlindFL supports diverse datasets and models efficiently whilst achieving robust privacy guarantees.
  arXiv  Detail & Related papers  (2022-06-16T07:26:50Z)
• Differentially Private AUC Computation in Vertical Federated Learning [21.692648490368327]
  We propose two evaluation algorithms that can more accurately compute the widely used AUC (area under curve) metric when using label DP in vFL. Through extensive experiments, we show our algorithms can achieve more accurate AUCs compared to the baselines.
  arXiv  Detail & Related papers  (2022-05-24T23:46:21Z)
• Desirable Companion for Vertical Federated Learning: New Zeroth-Order Gradient Based Algorithm [140.25480610981504]
  A complete list of metrics to evaluate VFL algorithms should include model applicability, privacy, communication, and computation efficiency. We propose a novel VFL framework with black-box scalability, which is inseparably inseparably scalable.
  arXiv  Detail & Related papers  (2022-03-19T13:55:47Z)
• Differentially private federated deep learning for multi-site medical image segmentation [56.30543374146002]
  Collaborative machine learning techniques such as federated learning (FL) enable the training of models on effectively larger datasets without data transfer. Recent initiatives have demonstrated that segmentation models trained with FL can achieve performance similar to locally trained models. However, FL is not a fully privacy-preserving technique and privacy-centred attacks can disclose confidential patient data.
  arXiv  Detail & Related papers  (2021-07-06T12:57:32Z)
• Voting-based Approaches For Differentially Private Federated Learning [87.2255217230752]
  This work is inspired by knowledge transfer non-federated privacy learning from Papernot et al. We design two new DPFL schemes, by voting among the data labels returned from each local model, instead of averaging the gradients. Our approaches significantly improve the privacy-utility trade-off over the state-of-the-arts in DPFL.
  arXiv  Detail & Related papers  (2020-10-09T23:55:19Z)
• User-Level Privacy-Preserving Federated Learning: Analysis and Performance Optimization [77.43075255745389]
  Federated learning (FL) is capable of preserving private data from mobile terminals (MTs) while training the data into useful models. From a viewpoint of information theory, it is still possible for a curious server to infer private information from the shared models uploaded by MTs. We propose a user-level differential privacy (UDP) algorithm by adding artificial noise to the shared models before uploading them to servers.
  arXiv  Detail & Related papers  (2020-02-29T10:13:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.