Fair Finetuning Mitigates Distribution Inference Attacks

• URL: http://arxiv.org/abs/2606.01719v1
• Date: Mon, 01 Jun 2026 05:34:19 GMT
• Title: Fair Finetuning Mitigates Distribution Inference Attacks
• Abstract summary: Machine learning models trained on sensitive data can inadvertently leak population-level information about their training distributions.<n>We propose Fair Fine-tuning (FFt): a trained model is fine-tuned on samples from a complementary distribution under an Equalized Odds constraint.<n>Our work provides the first formal bound connecting a model's measured EO disparity directly to its adversarial advantage in the DIA game.
• Score: 0.609170287691728
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine learning models trained on sensitive data can inadvertently leak population-level information about their training distributions -- a threat known as distribution inference attack (DIA). An adversary with black-box access can infer sensitive demographic properties, such as subgroup proportions, without observing any training data directly. While defenses such as differential privacy and property unlearning have been proposed, the link between fairness constraints and distributional leakage remains unexplored. We propose Fair Fine-tuning (FFt): a trained model is fine-tuned on samples from the complementary distribution under an Equalized Odds (EO) constraint. We provide a complete theoretical characterization, proving the tight bound $\text{Adv}(\mathcal{A},M_f) \le Δ_{\text{EO}} \cdot W$, where $W$ quantifies how distinguishable the two training distributions are by their sensitive-attribute composition. We also establish a necessary condition for FFt to reduce adversarial advantage and prove tightness of the bound. We evaluate across six datasets spanning tabular (ACS Income, COMPAS, German Credit), image (UTKFaces), and NLP (Bias in Bios) modalities. Rehearsal-based FFt consistently reduces the adversarial accuracy gap below the detection threshold $τ!=!0.1$ across all settings; on ACS Income, the gap falls from $\sim!15%$ to under $4%$. Our work provides the first formal bound connecting a model's measured EO disparity directly to its adversarial advantage in the DIA game, opening a new avenue for unified fairness-and-privacy defenses.

Related papers

• FairDiffuseVQVAE: Sampling-Time Fairness in Tabular Diffusion via Conditional Refinement of Vector-Quantized Latents [1.6444986611654075]
  We introduce FairDiffuseVQVAE, a two-stage architecture that decouples fidelity from fairness.<n>Fairness emerges as a property of the sampling distribution.<n>On the Adult, Bank and COMPAS datasets, FairDiffuseVQVAE achieves the highest mean Demographic Parity Ratio.
  arXiv  Detail & Related papers  (2026-07-31T01:57:07Z)
• Byzantine-Robust and Differentially Private Federated Optimization under Weaker Assumptions [89.52532304099522]
  Federated Learning (FL) enables clients to collaboratively train a shared model without centralizing their raw data, offering an inherent level of privacy.<n> gradients and model updates can still leak sensitive information, while malicious servers may mount adversarial attacks such as Byzantine manipulation.<n>These vulnerabilities highlight the need to address differential privacy (DP) and Byzantine robustness within a unified framework.<n>We propose Byz-Clip21-SGD2M, a new algorithm that integrates robust aggregation with double momentum and carefully designed clipping.
  arXiv  Detail & Related papers  (2026-03-24T17:39:09Z)
• Privacy-Preserving Federated Learning with Verifiable Fairness Guarantees [0.0]
  Federated learning enables collaborative model training across distributed institutions without centralizing sensitive data.<n>This paper introduces CryptoFair-FL, a novel cryptographic framework providing the first verifiable fairness guarantees for federated learning systems.
  arXiv  Detail & Related papers  (2026-01-18T15:06:30Z)
• GuardFed: A Trustworthy Federated Learning Framework Against Dual-Facet Attacks [56.983319121358555]
  Federated learning (FL) enables privacy-preserving collaborative model training but remains vulnerable to adversarial behaviors.<n>We introduce the Dual-Facet Attack (DFA), a novel threat model that concurrently undermines predictive accuracy and group fairness.<n>We propose GuardFed, a self-adaptive defense framework that maintains a fairness-aware reference model using a small amount of clean server data.
  arXiv  Detail & Related papers  (2025-11-12T13:02:45Z)
• Rejection via Learning Density Ratios [50.91522897152437]
  Classification with rejection emerges as a learning paradigm which allows models to abstain from making predictions.<n>We propose a different distributional perspective, where we seek to find an idealized data distribution which maximizes a pretrained model's performance.<n>Our framework is tested empirically over clean and noisy datasets.
  arXiv  Detail & Related papers  (2024-05-29T01:32:17Z)
• Minimax Optimal Fair Classification with Bounded Demographic Disparity [28.936244976415484]
  This paper explores the statistical foundations of fair binary classification with two protected groups. We show that using a finite sample incurs additional costs due to the need to estimate group-specific acceptance thresholds. We propose FairBayes-DDP+, a group-wise thresholding method with an offset that we show attains the minimax lower bound.
  arXiv  Detail & Related papers  (2024-03-27T02:59:04Z)
• Fairness Without Harm: An Influence-Guided Active Sampling Approach [32.173195437797766]
  We aim to train models that mitigate group fairness disparity without causing harm to model accuracy. The current data acquisition methods, such as fair active learning approaches, typically require annotating sensitive attributes. We propose a tractable active data sampling algorithm that does not rely on training group annotations.
  arXiv  Detail & Related papers  (2024-02-20T07:57:38Z)
• On the Vulnerability of Fairness Constrained Learning to Malicious Noise [28.176039923404883]
  We consider the vulnerability of fairness-constrained learning to small amounts of malicious noise in the training data. For example, for Demographic Parity we show we can incur only a $Theta(alpha)$ loss in accuracy, where $alpha$ is the malicious noise rate. For Equal Opportunity, we show we can incur an $O(sqrtalpha)$ loss, and give a matching $Omega(sqrtalpha)$ lower bound.
  arXiv  Detail & Related papers  (2023-07-21T20:26:54Z)
• Explicit Tradeoffs between Adversarial and Natural Distributional Robustness [48.44639585732391]
  In practice, models need to enjoy both types of robustness to ensure reliability. In this work, we show that in fact, explicit tradeoffs exist between adversarial and natural distributional robustness.
  arXiv  Detail & Related papers  (2022-09-15T19:58:01Z)
• The Power and Limitation of Pretraining-Finetuning for Linear Regression under Covariate Shift [127.21287240963859]
  We investigate a transfer learning approach with pretraining on the source data and finetuning based on the target data. For a large class of linear regression instances, transfer learning with $O(N2)$ source data is as effective as supervised learning with $N$ target data.
  arXiv  Detail & Related papers  (2022-08-03T05:59:49Z)
• Optimal Membership Inference Bounds for Adaptive Composition of Sampled Gaussian Mechanisms [93.44378960676897]
  Given a trained model and a data sample, membership-inference (MI) attacks predict whether the sample was in the model's training set. A common countermeasure against MI attacks is to utilize differential privacy (DP) during model training to mask the presence of individual examples. In this paper, we derive bounds for the textitadvantage of an adversary mounting a MI attack, and demonstrate tightness for the widely-used Gaussian mechanism.
  arXiv  Detail & Related papers  (2022-04-12T22:36:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.