Related papers: Privacy Cost as Equity Input: A Group Fairness Criterion for Differentially Private Machine Learning

Privacy Cost as Equity Input: A Group Fairness Criterion for Differentially Private Machine Learning

URL: http://arxiv.org/abs/2607.16620v1
Date: Sat, 18 Jul 2026 03:41:01 GMT
Title: Privacy Cost as Equity Input: A Group Fairness Criterion for Differentially Private Machine Learning
Abstract summary: We argue that privacy cost, the information leakage borne by each group, is itself a form of harm.<n>We derive a group fairness metric defined as a group's positive prediction rate normalized by its per-group overfitting gap.
Score: 0.609170287691728
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Differential privacy (DP) is increasingly deployed to limit membership inference risk in machine-learning systems. Prior work has shown that DP-SGD can widen accuracy disparities across demographic groups, but this framing treats fairness as a purely outcome-side concern. We argue that privacy cost, the information leakage borne by each group, is itself a form of harm, and adopt a compensatory-fairness framework in which a group that involuntarily bears greater privacy exposure is owed proportionally greater benefit from the system. From this principle we derive the \emph{Privacy-Cost Equity Ratio} (PCER), a group fairness metric defined as a group's positive prediction rate normalized by its per-group overfitting gap. By a standard membership inference bound, this overfitting gap upper-bounds each group's vulnerability to inference attacks, making PCER a conservative measure of benefit relative to exposure. PCER needs only per-group train and test accuracy (no shadow models), making it a practical post-hoc audit tool. We evaluate PCER alongside standard fairness metrics across six benchmark--attribute combinations spanning tabular and NLP domains, under DP-SGD at a range of privacy budgets, and validate the overfitting-gap proxy against a direct threshold membership-inference attack. The results reveal patterns that outcome-based metrics miss. On COMPAS, PCER uncovers a persistent double disadvantage: the protected group bears both greater privacy exposure and worse predictive outcomes, something demographic parity gap masks entirely. Sensitivity analysis shows very strong privacy guarantees collapse both groups' overfitting to a numerical floor, rendering exposure-based audits uninformative in that regime. Together, these findings show that fairness audits of privacy-preserving systems must account for who bears the cost of protection, not only who benefits from its outcomes.

Related papers

Auditing Fairness-Privacy Trade-offs: Subpopulation-Level Effects of Fairness-Enhancing Algorithms [9.177020525465272]
We present the first comprehensive study of how fairness interventions affect membership inference privacy risks at the subpopulation level.<n>Our results demonstrate that fairness interventions do not uniformly increase privacy risk; their impact depends on model architecture, subgroup size, and mitigation strategy.
arXiv Detail & Related papers (2026-07-16T06:10:42Z)
Your Privacy Depends on Others: Collusion Vulnerabilities in Individual Differential Privacy [50.66105844449181]
Individual Differential Privacy (iDP) promises users control over their privacy, but this promise can be broken in practice.<n>We reveal a previously overlooked vulnerability in sampling-based iDP mechanisms.<n>We propose $(varepsilon_i,_i,overline)$-iDP a privacy contract that uses $$-divergences to provide users with a hard upper bound on their excess vulnerability.
arXiv Detail & Related papers (2026-01-19T10:26:12Z)
Privacy-Preserving Federated Learning with Verifiable Fairness Guarantees [0.0]
Federated learning enables collaborative model training across distributed institutions without centralizing sensitive data.<n>This paper introduces CryptoFair-FL, a novel cryptographic framework providing the first verifiable fairness guarantees for federated learning systems.
arXiv Detail & Related papers (2026-01-18T15:06:30Z)
On the Fairness of Privacy Protection: Measuring and Mitigating the Disparity of Group Privacy Risks for Differentially Private Machine Learning [11.838077209919875]
We introduce a novel membership inference game that can efficiently audit the approximate worst-case privacy risks of data records.<n>Our algorithm effectively reduces the disparity in group privacy risks, thereby enhancing the fairness of privacy protection in DPML.
arXiv Detail & Related papers (2025-10-10T08:09:08Z)
Privacy Amplification for the Gaussian Mechanism via Bounded Support [64.86780616066575]
Data-dependent privacy accounting frameworks such as per-instance differential privacy (pDP) and Fisher information loss (FIL) confer fine-grained privacy guarantees for individuals in a fixed training dataset. We propose simple modifications of the Gaussian mechanism with bounded support, showing that they amplify privacy guarantees under data-dependent accounting.
arXiv Detail & Related papers (2024-03-07T21:22:07Z)
FairDP: Certified Fairness with Differential Privacy [55.51579601325759]
This paper introduces FairDP, a novel training mechanism designed to provide group fairness certification for the trained model's decisions.<n>The key idea of FairDP is to train models for distinct individual groups independently, add noise to each group's gradient for data privacy protection, and integrate knowledge from group models to formulate a model that balances privacy, utility, and fairness in downstream tasks.
arXiv Detail & Related papers (2023-05-25T21:07:20Z)
A Randomized Approach for Tight Privacy Accounting [63.67296945525791]
We propose a new differential privacy paradigm called estimate-verify-release (EVR) EVR paradigm first estimates the privacy parameter of a mechanism, then verifies whether it meets this guarantee, and finally releases the query output. Our empirical evaluation shows the newly proposed EVR paradigm improves the utility-privacy tradeoff for privacy-preserving machine learning.
arXiv Detail & Related papers (2023-04-17T00:38:01Z)
On the Alignment of Group Fairness with Attribute Privacy [1.6574413179773757]
Group fairness and privacy are fundamental aspects in designing trustworthy machine learning models. We are the first to demonstrate the alignment of group fairness with the specific privacy notion of attribute privacy in a blackbox setting.
arXiv Detail & Related papers (2022-11-18T13:00:34Z)
"You Can't Fix What You Can't Measure": Privately Measuring Demographic Performance Disparities in Federated Learning [78.70083858195906]
We propose differentially private mechanisms to measure differences in performance across groups while protecting the privacy of group membership. Our results show that, contrary to what prior work suggested, protecting privacy is not necessarily in conflict with identifying performance disparities of federated models.
arXiv Detail & Related papers (2022-06-24T09:46:43Z)
The Impact of Differential Privacy on Group Disparity Mitigation [28.804933301007644]
We evaluate the impact of differential privacy on fairness across four tasks. We train $(varepsilon,delta)$-differentially private models with empirical risk minimization. We find that differential privacy increases between-group performance differences in the baseline setting. But differential privacy reduces between-group performance differences in the robust setting.
arXiv Detail & Related papers (2022-03-05T13:55:05Z)
Private Reinforcement Learning with PAC and Regret Guarantees [69.4202374491817]
We design privacy preserving exploration policies for episodic reinforcement learning (RL) We first provide a meaningful privacy formulation using the notion of joint differential privacy (JDP) We then develop a private optimism-based learning algorithm that simultaneously achieves strong PAC and regret bounds, and enjoys a JDP guarantee.
arXiv Detail & Related papers (2020-09-18T20:18:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.