DP-SGD vs PATE: Which Has Less Disparate Impact on Model Accuracy?

• URL: http://arxiv.org/abs/2106.12576v1
• Date: Tue, 22 Jun 2021 20:37:12 GMT
• Title: DP-SGD vs PATE: Which Has Less Disparate Impact on Model Accuracy?
• Authors: Archit Uniyal, Rakshit Naidu, Sasikanth Kotti, Sahib Singh, Patrik Joslin Kenfack, Fatemehsadat Mireshghallah, Andrew Trask
• Abstract summary: We show that application of differential privacy, specifically the DP-SGD algorithm, has a disparate impact on different sub-groups in the population. We compare PATE, another mechanism for training deep learning models using differential privacy, with DP-SGD in terms of fairness.
• Score: 1.3238373064156095
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in differentially private deep learning have demonstrated that application of differential privacy, specifically the DP-SGD algorithm, has a disparate impact on different sub-groups in the population, which leads to a significantly high drop-in model utility for sub-populations that are under-represented (minorities), compared to well-represented ones. In this work, we aim to compare PATE, another mechanism for training deep learning models using differential privacy, with DP-SGD in terms of fairness. We show that PATE does have a disparate impact too, however, it is much less severe than DP-SGD. We draw insights from this observation on what might be promising directions in achieving better fairness-privacy trade-offs.

Related papers

• Does Differential Privacy Impact Bias in Pretrained NLP Models? [24.63118058112066]
  Differential privacy (DP) is applied when fine-tuning pre-trained large language models (LLMs) to limit leakage of training examples. We show the impact of DP on bias in LLMs through empirical analysis. Our results also show that the impact of DP on bias is not only affected by the privacy protection level but also the underlying distribution of the dataset.
  arXiv  Detail & Related papers  (2024-10-24T13:59:03Z)
• Pre-training Differentially Private Models with Limited Public Data [54.943023722114134]
  differential privacy (DP) is a prominent method to gauge the degree of security provided to the models. DP is yet not capable of protecting a substantial portion of the data used during the initial pre-training stage. We develop a novel DP continual pre-training strategy using only 10% of public data. Our strategy can achieve DP accuracy of 41.5% on ImageNet-21k, as well as non-DP accuracy of 55.7% and and 60.0% on downstream tasks Places365 and iNaturalist-2021.
  arXiv  Detail & Related papers  (2024-02-28T23:26:27Z)
• De-amplifying Bias from Differential Privacy in Language Model Fine-tuning [10.847913815093179]
  Fairness and privacy are two important values machine learning (ML) practitioners often seek to operationalize in models. We show that DP amplifies gender, racial, and religious bias when fine-tuning large language models. We demonstrate that Counterfactual Data Augmentation, a known method for addressing bias, also mitigates bias amplification by DP.
  arXiv  Detail & Related papers  (2024-02-07T00:30:58Z)
• Sparsity-Preserving Differentially Private Training of Large Embedding Models [67.29926605156788]
  DP-SGD is a training algorithm that combines differential privacy with gradient descent. Applying DP-SGD naively to embedding models can destroy gradient sparsity, leading to reduced training efficiency. We present two new algorithms, DP-FEST and DP-AdaFEST, that preserve gradient sparsity during private training of large embedding models.
  arXiv  Detail & Related papers  (2023-11-14T17:59:51Z)
• DP-SGD for non-decomposable objective functions [0.0]
  We develop a new variant for similarity based loss functions that manipulates gradients of the objective function in a novel way to obtain a senstivity of the summed gradient that is $O(1)$ for batch size $n$. Our method's performance comes close to that of a non-private model and generally outperforms DP-SGD applied directly to the contrastive loss.
  arXiv  Detail & Related papers  (2023-10-04T18:48:16Z)
• Bias-Aware Minimisation: Understanding and Mitigating Estimator Bias in Private SGD [56.01810892677744]
  We show a connection between per-sample gradient norms and the estimation bias of the private gradient oracle used in DP-SGD. We propose Bias-Aware Minimisation (BAM) that allows for the provable reduction of private gradient estimator bias.
  arXiv  Detail & Related papers  (2023-08-23T09:20:41Z)
• Private Ad Modeling with DP-SGD [58.670969449674395]
  A well-known algorithm in privacy-preserving ML is differentially private gradient descent (DP-SGD) In this work we apply DP-SGD to several ad modeling tasks including predicting click-through rates, conversion rates, and number of conversion events. Our work is the first to empirically demonstrate that DP-SGD can provide both privacy and utility for ad modeling tasks.
  arXiv  Detail & Related papers  (2022-11-21T22:51:16Z)
• Large Scale Transfer Learning for Differentially Private Image Classification [51.10365553035979]
  Differential Privacy (DP) provides a formal framework for training machine learning models with individual example level privacy. Private training using DP-SGD protects against leakage by injecting noise into individual example gradients. While this result is quite appealing, the computational cost of training large-scale models with DP-SGD is substantially higher than non-private training.
  arXiv  Detail & Related papers  (2022-05-06T01:22:20Z)
• DP-SGD vs PATE: Which Has Less Disparate Impact on GANs? [0.0]
  We compare GANs trained with the two best-known DP frameworks for deep learning, DP-SGD, and PATE, in different data imbalance settings. Our experiments consistently show that for PATE, unlike DP-SGD, the privacy-utility trade-off is not monotonically decreasing.
  arXiv  Detail & Related papers  (2021-11-26T17:25:46Z)
• Differentially Private Federated Learning with Laplacian Smoothing [72.85272874099644]
  Federated learning aims to protect data privacy by collaboratively learning a model without sharing private data among users. An adversary may still be able to infer the private training data by attacking the released model. Differential privacy provides a statistical protection against such attacks at the price of significantly degrading the accuracy or utility of the trained models.
  arXiv  Detail & Related papers  (2020-05-01T04:28:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.