fairBERTs: Erasing Sensitive Information Through Semantic and Fairness-aware Perturbations

• URL: http://arxiv.org/abs/2407.08189v1
• Date: Thu, 11 Jul 2024 05:13:38 GMT
• Title: fairBERTs: Erasing Sensitive Information Through Semantic and Fairness-aware Perturbations
• Abstract summary: We present fairBERTs, a framework for learning fair fine-tuned BERT series models. We demonstrate the great superiority of fairBERTs in mitigating unfairness while maintaining the model utility.
• Score: 35.60919203938758
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Pre-trained language models (PLMs) have revolutionized both the natural language processing research and applications. However, stereotypical biases (e.g., gender and racial discrimination) encoded in PLMs have raised negative ethical implications for PLMs, which critically limits their broader applications. To address the aforementioned unfairness issues, we present fairBERTs, a general framework for learning fair fine-tuned BERT series models by erasing the protected sensitive information via semantic and fairness-aware perturbations generated by a generative adversarial network. Through extensive qualitative and quantitative experiments on two real-world tasks, we demonstrate the great superiority of fairBERTs in mitigating unfairness while maintaining the model utility. We also verify the feasibility of transferring adversarial components in fairBERTs to other conventionally trained BERT-like models for yielding fairness improvements. Our findings may shed light on further research on building fairer fine-tuned PLMs.

Related papers

• Addressing Stereotypes in Large Language Models: A Critical Examination and Mitigation [0.0]
  Large Language models (LLMs) have gained popularity in recent years with the advancement of Natural Language Processing (NLP)<n>This study inspects and highlights the need to address biases in LLMs amid growing generative Artificial Intelligence (AI)<n>We utilize bias-specific benchmarks such StereoSet and CrowSPairs to evaluate the existence of various biases in many different generative models such as BERT, GPT 3.5, and ADA.
  arXiv  Detail & Related papers  (2025-11-18T05:43:34Z)
• A Comprehensive Study of Implicit and Explicit Biases in Large Language Models [1.0555164678638427]
  This study highlights the need to address biases in Large Language Models amid growing generative AI.<n>We studied bias-specific benchmarks such as StereoSet and CrowSPairs to evaluate the existence of various biases in multiple generative models such as BERT and GPT 3.5.<n>Results indicated fine-tuned models struggle with gender biases but excelled at identifying and avoiding racial biases.
  arXiv  Detail & Related papers  (2025-11-18T05:27:17Z)
• Learning Fair Representations with Kolmogorov-Arnold Networks [0.08594140167290099]
  Predictive models often exhibit discriminatory behavior towards marginalized groups.<n>Existing fair learning models aim to mitigate bias, but achieving an optimal trade-off between fairness and accuracy remains a challenge.<n>We propose integrating Kolmogorov-Arnold Networks (KANs) within a fair adversarial learning framework.
  arXiv  Detail & Related papers  (2025-11-14T07:51:56Z)
• Detecting and Mitigating Bias in LLMs through Knowledge Graph-Augmented Training [2.8402080392117757]
  This work investigates Knowledge Graph-Augmented Training (KGAT) as a novel method to mitigate bias in large language models. Public datasets for bias assessment include Gender Shades, Bias in Bios, and FairFace. We also performed targeted mitigation strategies to correct biased associations, leading to a significant drop in biased output and improved bias metrics.
  arXiv  Detail & Related papers  (2025-04-01T00:27:50Z)
• On the Fairness, Diversity and Reliability of Text-to-Image Generative Models [68.62012304574012]
  multimodal generative models have sparked critical discussions on their reliability, fairness and potential for misuse.<n>We propose an evaluation framework to assess model reliability by analyzing responses to global and local perturbations in the embedding space.<n>Our method lays the groundwork for detecting unreliable, bias-injected models and tracing the provenance of embedded biases.
  arXiv  Detail & Related papers  (2024-11-21T09:46:55Z)
• Editable Fairness: Fine-Grained Bias Mitigation in Language Models [52.66450426729818]
  We propose a novel debiasing approach, Fairness Stamp (FAST), which enables fine-grained calibration of individual social biases. FAST surpasses state-of-the-art baselines with superior debiasing performance. This highlights the potential of fine-grained debiasing strategies to achieve fairness in large language models.
  arXiv  Detail & Related papers  (2024-08-07T17:14:58Z)
• Are Large Language Models Really Bias-Free? Jailbreak Prompts for Assessing Adversarial Robustness to Bias Elicitation [0.0]
  Large Language Models (LLMs) have revolutionized artificial intelligence, demonstrating remarkable computational power and linguistic capabilities. These models are inherently prone to various biases stemming from their training data. This study explores the presence of these biases within the responses given by the most recent LLMs, analyzing the impact on their fairness and reliability.
  arXiv  Detail & Related papers  (2024-07-11T12:30:19Z)
• FairPFN: Transformers Can do Counterfactual Fairness [41.052676173417574]
  Causal and counterfactual fairness provides an intuitive way to define fairness that closely aligns with legal standards. This study builds upon recent work in in context learning (ICL) and prior fitted networks (PFNs) to learn a transformer called FairPFN. This model is pretrained using synthetic fairness data to eliminate the causal effects of protected attributes directly from observational data.
  arXiv  Detail & Related papers  (2024-07-08T08:36:44Z)
• LIDAO: Towards Limited Interventions for Debiasing (Large) Language Models [19.18522268167047]
  Large language models (LLMs) have achieved impressive performance on various natural language generation tasks. However, they suffer from generating negative and harmful contents that are biased against certain demographic groups. We propose LIDAO, a framework to debias a (L)LM at a better fluency provably.
  arXiv  Detail & Related papers  (2024-06-01T20:12:54Z)
• Learning for Counterfactual Fairness from Observational Data [62.43249746968616]
  Fairness-aware machine learning aims to eliminate biases of learning models against certain subgroups described by certain protected (sensitive) attributes such as race, gender, and age. A prerequisite for existing methods to achieve counterfactual fairness is the prior human knowledge of the causal model for the data. In this work, we address the problem of counterfactually fair prediction from observational data without given causal models by proposing a novel framework CLAIRE.
  arXiv  Detail & Related papers  (2023-07-17T04:08:29Z)
• DualFair: Fair Representation Learning at Both Group and Individual Levels via Contrastive Self-supervision [73.80009454050858]
  This work presents a self-supervised model, called DualFair, that can debias sensitive attributes like gender and race from learned representations. Our model jointly optimize for two fairness criteria - group fairness and counterfactual fairness.
  arXiv  Detail & Related papers  (2023-03-15T07:13:54Z)
• Large Language Models with Controllable Working Memory [64.71038763708161]
  Large language models (LLMs) have led to a series of breakthroughs in natural language processing (NLP) What further sets these models apart is the massive amounts of world knowledge they internalize during pretraining. How the model's world knowledge interacts with the factual information presented in the context remains under explored.
  arXiv  Detail & Related papers  (2022-11-09T18:58:29Z)
• COFFEE: Counterfactual Fairness for Personalized Text Generation in Explainable Recommendation [56.520470678876656]
  bias inherent in user written text can associate different levels of linguistic quality with users' protected attributes. We introduce a general framework to achieve measure-specific counterfactual fairness in explanation generation.
  arXiv  Detail & Related papers  (2022-10-14T02:29:10Z)
• Developing a novel fair-loan-predictor through a multi-sensitive debiasing pipeline: DualFair [2.149265948858581]
  We create a novel bias mitigation technique called DualFair and develop a new fairness metric (i.e. AWI) that can handle MSPSO. We test our novel mitigation method using a comprehensive U.S mortgage lending dataset and show that our classifier, or fair loan predictor, obtains better fairness and accuracy metrics than current state-of-the-art models.
  arXiv  Detail & Related papers  (2021-10-17T23:13:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.