Neither Private Nor Fair: Impact of Data Imbalance on Utility and Fairness in Differential Privacy

• URL: http://arxiv.org/abs/2009.06389v3
• Date: Sat, 3 Oct 2020 11:55:05 GMT
• Title: Neither Private Nor Fair: Impact of Data Imbalance on Utility and Fairness in Differential Privacy
• Authors: Tom Farrand, Fatemehsadat Mireshghallah, Sahib Singh, Andrew Trask
• Abstract summary: We study how different levels of imbalance in the data affect the accuracy and the fairness of the decisions made by the model. We demonstrate that even small imbalances and loose privacy guarantees can cause disparate impacts.
• Score: 5.416049433853457
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deployment of deep learning in different fields and industries is growing day by day due to its performance, which relies on the availability of data and compute. Data is often crowd-sourced and contains sensitive information about its contributors, which leaks into models that are trained on it. To achieve rigorous privacy guarantees, differentially private training mechanisms are used. However, it has recently been shown that differential privacy can exacerbate existing biases in the data and have disparate impacts on the accuracy of different subgroups of data. In this paper, we aim to study these effects within differentially private deep learning. Specifically, we aim to study how different levels of imbalance in the data affect the accuracy and the fairness of the decisions made by the model, given different levels of privacy. We demonstrate that even small imbalances and loose privacy guarantees can cause disparate impacts.

Related papers

• A Systematic and Formal Study of the Impact of Local Differential Privacy on Fairness: Preliminary Results [5.618541935188389]
  Differential privacy (DP) is the predominant solution for privacy-preserving Machine learning (ML) algorithms. Recent experimental studies have shown that local DP can impact ML prediction for different subgroups of individuals. We study how the fairness of the decisions made by the ML model changes under local DP for different levels of privacy and data distributions.
  arXiv  Detail & Related papers  (2024-05-23T15:54:03Z)
• FewFedPIT: Towards Privacy-preserving and Few-shot Federated Instruction Tuning [54.26614091429253]
  Federated instruction tuning (FedIT) is a promising solution, by consolidating collaborative training across multiple data owners. FedIT encounters limitations such as scarcity of instructional data and risk of exposure to training data extraction attacks. We propose FewFedPIT, designed to simultaneously enhance privacy protection and model performance of federated few-shot learning.
  arXiv  Detail & Related papers  (2024-03-10T08:41:22Z)
• Federated Learning with Differential Privacy [0.9208007322096533]
  Federated learning (FL) is capable of preserving client's private data from being shared among different parties. Private information can still be divulged by analyzing uploaded parameter weights from clients. Our results show that non-i.i.d and small datasets have the highest decrease in performance in a distributed and differentially private setting.
  arXiv  Detail & Related papers  (2024-02-03T18:21:38Z)
• Delete My Account: Impact of Data Deletion on Machine Learning Classifiers [0.0]
  The right to erasure has potential implications for a number of different fields, such as big data and machine learning. Our paper presents an in-depth analysis about the impact of the use of the right to erasure on the performance of machine learning models.
  arXiv  Detail & Related papers  (2023-11-17T08:23:17Z)
• Causal Inference with Differentially Private (Clustered) Outcomes [16.166525280886578]
  Estimating causal effects from randomized experiments is only feasible if participants agree to reveal their responses. We suggest a new differential privacy mechanism, Cluster-DP, which leverages any given cluster structure. We show that, depending on an intuitive measure of cluster quality, we can improve the variance loss while maintaining our privacy guarantees.
  arXiv  Detail & Related papers  (2023-08-02T05:51:57Z)
• Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining [75.25943383604266]
  We question whether the use of large Web-scraped datasets should be viewed as differential-privacy-preserving. We caution that publicizing these models pretrained on Web data as "private" could lead to harm and erode the public's trust in differential privacy as a meaningful definition of privacy. We conclude by discussing potential paths forward for the field of private learning, as public pretraining becomes more popular and powerful.
  arXiv  Detail & Related papers  (2022-12-13T10:41:12Z)
• The Privacy Onion Effect: Memorization is Relative [76.46529413546725]
  We show an Onion Effect of memorization: removing the "layer" of outlier points that are most vulnerable exposes a new layer of previously-safe points to the same attack. It suggests that privacy-enhancing technologies such as machine unlearning could actually harm the privacy of other users.
  arXiv  Detail & Related papers  (2022-06-21T15:25:56Z)
• Post-processing of Differentially Private Data: A Fairness Perspective [53.29035917495491]
  This paper shows that post-processing causes disparate impacts on individuals or groups. It analyzes two critical settings: the release of differentially private datasets and the use of such private datasets for downstream decisions. It proposes a novel post-processing mechanism that is (approximately) optimal under different fairness metrics.
  arXiv  Detail & Related papers  (2022-01-24T02:45:03Z)
• Differentially Private Deep Learning under the Fairness Lens [34.28936739262812]
  Differential Privacy (DP) is an important privacy-enhancing technology for private machine learning systems. It allows to measure and bound the risk associated with an individual participation in a computation. It was recently observed that DP learning systems may exacerbate bias and unfairness for different groups of individuals.
  arXiv  Detail & Related papers  (2021-06-04T19:10:09Z)
• Robustness Threats of Differential Privacy [70.818129585404]
  We experimentally demonstrate that networks, trained with differential privacy, in some settings might be even more vulnerable in comparison to non-private versions. We study how the main ingredients of differentially private neural networks training, such as gradient clipping and noise addition, affect the robustness of the model.
  arXiv  Detail & Related papers  (2020-12-14T18:59:24Z)
• Differentially Private and Fair Deep Learning: A Lagrangian Dual Approach [54.32266555843765]
  This paper studies a model that protects the privacy of the individuals sensitive information while also allowing it to learn non-discriminatory predictors. The method relies on the notion of differential privacy and the use of Lagrangian duality to design neural networks that can accommodate fairness constraints.
  arXiv  Detail & Related papers  (2020-09-26T10:50:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.