Fair NLP Models with Differentially Private Text Encoders

• URL: http://arxiv.org/abs/2205.06135v1
• Date: Thu, 12 May 2022 14:58:38 GMT
• Title: Fair NLP Models with Differentially Private Text Encoders
• Authors: Gaurav Maheshwari, Pascal Denis, Mikaela Keller, Aur\'elien Bellet
• Abstract summary: We propose FEDERATE, an approach that combines ideas from differential privacy and adversarial training to learn private text representations. We empirically evaluate the trade-off between the privacy of the representations and the fairness and accuracy of the downstream model on four NLP datasets.
• Score: 1.7434507809930746
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Encoded text representations often capture sensitive attributes about individuals (e.g., race or gender), which raise privacy concerns and can make downstream models unfair to certain groups. In this work, we propose FEDERATE, an approach that combines ideas from differential privacy and adversarial training to learn private text representations which also induces fairer models. We empirically evaluate the trade-off between the privacy of the representations and the fairness and accuracy of the downstream model on four NLP datasets. Our results show that FEDERATE consistently improves upon previous methods, and thus suggest that privacy and fairness can positively reinforce each other.

Related papers

• Private prediction for large-scale synthetic text generation [28.488459921169905]
  We present an approach for generating differentially private synthetic text using large language models (LLMs) In the private prediction framework, we only require the output synthetic data to satisfy differential privacy guarantees.
  arXiv  Detail & Related papers  (2024-07-16T18:28:40Z)
• Federated Face Forgery Detection Learning with Personalized Representation [63.90408023506508]
  Deep generator technology can produce high-quality fake videos that are indistinguishable, posing a serious social threat. Traditional forgery detection methods directly centralized training on data. The paper proposes a novel federated face forgery detection learning with personalized representation.
  arXiv  Detail & Related papers  (2024-06-17T02:20:30Z)
• PrivacyMind: Large Language Models Can Be Contextual Privacy Protection Learners [81.571305826793]
  We introduce Contextual Privacy Protection Language Models (PrivacyMind) Our work offers a theoretical analysis for model design and benchmarks various techniques. In particular, instruction tuning with both positive and negative examples stands out as a promising method.
  arXiv  Detail & Related papers  (2023-10-03T22:37:01Z)
• Independent Distribution Regularization for Private Graph Embedding [55.24441467292359]
  Graph embeddings are susceptible to attribute inference attacks, which allow attackers to infer private node attributes from the learned graph embeddings. To address these concerns, privacy-preserving graph embedding methods have emerged. We propose a novel approach called Private Variational Graph AutoEncoders (PVGAE) with the aid of independent distribution penalty as a regularization term.
  arXiv  Detail & Related papers  (2023-08-16T13:32:43Z)
• Differentially Private Language Models for Secure Data Sharing [19.918137395199224]
  In this paper, we show how to train a generative language model in a differentially private manner and consequently sampling data from it. Using natural language prompts and a new prompt-mismatch loss, we are able to create highly accurate and fluent textual datasets. We perform thorough experiments indicating that our synthetic datasets do not leak information from our original data and are of high language quality.
  arXiv  Detail & Related papers  (2022-10-25T11:12:56Z)
• FairVFL: A Fair Vertical Federated Learning Framework with Contrastive Adversarial Learning [102.92349569788028]
  We propose a fair vertical federated learning framework (FairVFL) to improve the fairness of VFL models. The core idea of FairVFL is to learn unified and fair representations of samples based on the decentralized feature fields in a privacy-preserving way. For protecting user privacy, we propose a contrastive adversarial learning method to remove private information from the unified representation in server.
  arXiv  Detail & Related papers  (2022-06-07T11:43:32Z)
• Just Fine-tune Twice: Selective Differential Privacy for Large Language Models [69.66654761324702]
  We propose a simple yet effective just-fine-tune-twice privacy mechanism to achieve SDP for large Transformer-based language models. Experiments show that our models achieve strong performance while staying robust to the canary insertion attack.
  arXiv  Detail & Related papers  (2022-04-15T22:36:55Z)
• SF-PATE: Scalable, Fair, and Private Aggregation of Teacher Ensembles [50.90773979394264]
  This paper studies a model that protects the privacy of individuals' sensitive information while also allowing it to learn non-discriminatory predictors. A key characteristic of the proposed model is to enable the adoption of off-the-selves and non-private fair models to create a privacy-preserving and fair model.
  arXiv  Detail & Related papers  (2022-04-11T14:42:54Z)
• Why Should I Trust a Model is Private? Using Shifts in Model Explanation for Evaluating Privacy-Preserving Emotion Recognition Model [35.016050900061]
  We focus on using interpretable methods to evaluate a model's efficacy to preserve privacy with respect to sensitive variables. We show how certain commonly-used methods that seek to preserve privacy might not align with human perception of privacy preservation. We conduct crowdsourcing experiments to evaluate the inclination of the evaluators to choose a particular model for a given task.
  arXiv  Detail & Related papers  (2021-04-18T09:56:41Z)
• On the Privacy Risks of Algorithmic Fairness [9.429448411561541]
  We study the privacy risks of group fairness through the lens of membership inference attacks. We show that fairness comes at the cost of privacy, and this cost is not distributed equally.
  arXiv  Detail & Related papers  (2020-11-07T09:15:31Z)
• Differentially Private Representation for NLP: Formal Guarantee and An Empirical Study on Privacy and Fairness [38.90014773292902]
  It has been demonstrated that hidden representation learned by a deep model can encode private information of the input. We propose Differentially Private Neural Representation (DPNR) to preserve the privacy of the extracted representation from text.
  arXiv  Detail & Related papers  (2020-10-03T05:58:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.