Privacy at a Price: Exploring its Dual Impact on AI Fairness

• URL: http://arxiv.org/abs/2404.09391v1
• Date: Mon, 15 Apr 2024 00:23:41 GMT
• Title: Privacy at a Price: Exploring its Dual Impact on AI Fairness
• Authors: Mengmeng Yang, Ming Ding, Youyang Qu, Wei Ni, David Smith, Thierry Rakotoarivelo,
• Abstract summary: We show that differential privacy in machine learning models can unequally impact separate demographic subgroups regarding prediction accuracy. This leads to a fairness concern, and manifests as biased performance. implementing gradient clipping in the differentially private gradient descent ML method can mitigate the negative impact of DP noise on fairness.
• Score: 24.650648702853903
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The worldwide adoption of machine learning (ML) and deep learning models, particularly in critical sectors, such as healthcare and finance, presents substantial challenges in maintaining individual privacy and fairness. These two elements are vital to a trustworthy environment for learning systems. While numerous studies have concentrated on protecting individual privacy through differential privacy (DP) mechanisms, emerging research indicates that differential privacy in machine learning models can unequally impact separate demographic subgroups regarding prediction accuracy. This leads to a fairness concern, and manifests as biased performance. Although the prevailing view is that enhancing privacy intensifies fairness disparities, a smaller, yet significant, subset of research suggests the opposite view. In this article, with extensive evaluation results, we demonstrate that the impact of differential privacy on fairness is not monotonous. Instead, we observe that the accuracy disparity initially grows as more DP noise (enhanced privacy) is added to the ML process, but subsequently diminishes at higher privacy levels with even more noise. Moreover, implementing gradient clipping in the differentially private stochastic gradient descent ML method can mitigate the negative impact of DP noise on fairness. This mitigation is achieved by moderating the disparity growth through a lower clipping threshold.

Related papers

• Masked Differential Privacy [64.32494202656801]
  We propose an effective approach called masked differential privacy (DP), which allows for controlling sensitive regions where differential privacy is applied. Our method operates selectively on data and allows for defining non-sensitive-temporal regions without DP application or combining differential privacy with other privacy techniques within data samples.
  arXiv  Detail & Related papers  (2024-10-22T15:22:53Z)
• A Systematic and Formal Study of the Impact of Local Differential Privacy on Fairness: Preliminary Results [5.618541935188389]
  Differential privacy (DP) is the predominant solution for privacy-preserving Machine learning (ML) algorithms. Recent experimental studies have shown that local DP can impact ML prediction for different subgroups of individuals. We study how the fairness of the decisions made by the ML model changes under local DP for different levels of privacy and data distributions.
  arXiv  Detail & Related papers  (2024-05-23T15:54:03Z)
• TernaryVote: Differentially Private, Communication Efficient, and Byzantine Resilient Distributed Optimization on Heterogeneous Data [50.797729676285876]
  We propose TernaryVote, which combines a ternary compressor and the majority vote mechanism to realize differential privacy, gradient compression, and Byzantine resilience simultaneously. We theoretically quantify the privacy guarantee through the lens of the emerging f-differential privacy (DP) and the Byzantine resilience of the proposed algorithm.
  arXiv  Detail & Related papers  (2024-02-16T16:41:14Z)
• Training Private Models That Know What They Don't Know [40.19666295972155]
  We find that several popular selective prediction approaches are ineffective in a differentially private setting. We propose a novel evaluation mechanism which isolate selective prediction performance across model utility levels.
  arXiv  Detail & Related papers  (2023-05-28T12:20:07Z)
• "You Can't Fix What You Can't Measure": Privately Measuring Demographic Performance Disparities in Federated Learning [78.70083858195906]
  We propose differentially private mechanisms to measure differences in performance across groups while protecting the privacy of group membership. Our results show that, contrary to what prior work suggested, protecting privacy is not necessarily in conflict with identifying performance disparities of federated models.
  arXiv  Detail & Related papers  (2022-06-24T09:46:43Z)
• Mixed Differential Privacy in Computer Vision [133.68363478737058]
  AdaMix is an adaptive differentially private algorithm for training deep neural network classifiers using both private and public image data. A few-shot or even zero-shot learning baseline that ignores private data can outperform fine-tuning on a large private dataset.
  arXiv  Detail & Related papers  (2022-03-22T06:15:43Z)
• The Impact of Differential Privacy on Group Disparity Mitigation [28.804933301007644]
  We evaluate the impact of differential privacy on fairness across four tasks. We train $(varepsilon,delta)$-differentially private models with empirical risk minimization. We find that differential privacy increases between-group performance differences in the baseline setting. But differential privacy reduces between-group performance differences in the robust setting.
  arXiv  Detail & Related papers  (2022-03-05T13:55:05Z)
• Differentially Private Deep Learning under the Fairness Lens [34.28936739262812]
  Differential Privacy (DP) is an important privacy-enhancing technology for private machine learning systems. It allows to measure and bound the risk associated with an individual participation in a computation. It was recently observed that DP learning systems may exacerbate bias and unfairness for different groups of individuals.
  arXiv  Detail & Related papers  (2021-06-04T19:10:09Z)
• Gradient Masking and the Underestimated Robustness Threats of Differential Privacy in Deep Learning [0.0]
  This paper experimentally evaluates the impact of training with Differential Privacy (DP) on model vulnerability against a broad range of adversarial attacks. The results suggest that private models are less robust than their non-private counterparts, and that adversarial examples transfer better among DP models than between non-private and private ones.
  arXiv  Detail & Related papers  (2021-05-17T16:10:54Z)
• Robustness Threats of Differential Privacy [70.818129585404]
  We experimentally demonstrate that networks, trained with differential privacy, in some settings might be even more vulnerable in comparison to non-private versions. We study how the main ingredients of differentially private neural networks training, such as gradient clipping and noise addition, affect the robustness of the model.
  arXiv  Detail & Related papers  (2020-12-14T18:59:24Z)
• Differentially Private and Fair Deep Learning: A Lagrangian Dual Approach [54.32266555843765]
  This paper studies a model that protects the privacy of the individuals sensitive information while also allowing it to learn non-discriminatory predictors. The method relies on the notion of differential privacy and the use of Lagrangian duality to design neural networks that can accommodate fairness constraints.
  arXiv  Detail & Related papers  (2020-09-26T10:50:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.