TriCon-Fair: Triplet Contrastive Learning for Mitigating Social Bias in Pre-trained Language Models

• URL: http://arxiv.org/abs/2511.00854v1
• Date: Sun, 02 Nov 2025 08:37:56 GMT
• Title: TriCon-Fair: Triplet Contrastive Learning for Mitigating Social Bias in Pre-trained Language Models
• Abstract summary: TriCon-Fair is a contrastive learning framework that employs a decoupled loss that combines triplet and language modeling terms to eliminate positive-negative coupling.<n> Experimental results demonstrate that TriCon-Fair reduces discriminatory output beyond existing debiasing baselines.
• Score: 14.709063635156054
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The increasing utilization of large language models raises significant concerns about the propagation of social biases, which may result in harmful and unfair outcomes. However, existing debiasing methods treat the biased and unbiased samples independently, thus ignoring their mutual relationship. This oversight enables a hidden negative-positive coupling, where improvements for one group inadvertently compromise the other, allowing residual social bias to persist. In this paper, we introduce TriCon-Fair, a contrastive learning framework that employs a decoupled loss that combines triplet and language modeling terms to eliminate positive-negative coupling. Our TriCon-Fair assigns each anchor an explicitly biased negative and an unbiased positive, decoupling the push-pull dynamics and avoiding positive-negative coupling, and jointly optimizes a language modeling (LM) objective to preserve general capability. Experimental results demonstrate that TriCon-Fair reduces discriminatory output beyond existing debiasing baselines while maintaining strong downstream performance. This suggests that our proposed TriCon-Fair offers a practical and ethical solution for sensitive NLP applications.

Related papers

• ResRL: Boosting LLM Reasoning via Negative Sample Projection Residual Reinforcement Learning [60.411097231348975]
  Reinforcement Learning with Verifiable Rewards (RLVR) enhances reasoning of Large Language Models (LLMs)<n>This paper proposes negative sample projection Residual Reinforcement Learning (ResRL) that decouples similar semantic distributions among positive and negative responses.
  arXiv  Detail & Related papers  (2026-05-01T03:57:44Z)
• Via Negativa for AI Alignment: Why Negative Constraints Are Structurally Superior to Positive Preferences [0.786119752211706]
  Training large language models with negative-only feedback can match or exceed standard reinforcement learning from human feedback.<n>No unified theoretical account explains why negative signals are so effective.<n>Positive preferences and negative constraints are structurally asymmetric.
  arXiv  Detail & Related papers  (2026-03-17T11:52:18Z)
• The Unfairness of Multifactorial Bias in Recommendation [68.35079031029616]
  Popularity bias and positivity bias are prominent sources of bias in recommender systems.<n>In this work, we examine how multifactorial bias influences item-side fairness.<n>We adapt a percentile-based rating transformation as a pre-processing strategy to mitigate multifactorial bias.
  arXiv  Detail & Related papers  (2026-01-19T08:37:43Z)
• Bias Is a Subspace, Not a Coordinate: A Geometric Rethinking of Post-hoc Debiasing in Vision-Language Models [13.46898179372249]
  Vision-Language Models (VLMs) have become indispensable for multimodal reasoning, yet their representations often encode and amplify demographic biases.<n>We propose a geometrically principled framework that identifies and removes the entire subspace of linearly decodable bias.<n>Our method achieves more robust debiasing with an average improvement of $18.5%$ across four fairness metrics.
  arXiv  Detail & Related papers  (2025-11-22T17:04:30Z)
• KLAAD: Refining Attention Mechanisms to Reduce Societal Bias in Generative Language Models [1.649505438157608]
  Large language models (LLMs) often exhibit societal biases in their outputs, prompting ethical concerns regarding fairness and harm.<n>We propose KLAAD (KL-Attention Alignment Debiasing), an attention-based debiasing framework that implicitly aligns attention distributions between stereotypical and anti-stereotypical sentence pairs.<n> Experimental evaluation of KLAAD demonstrates improved bias mitigation on both the BBQ and BOLD benchmarks, with minimal impact on language modeling quality.
  arXiv  Detail & Related papers  (2025-07-26T14:24:19Z)
• Paying Alignment Tax with Contrastive Learning [6.232983467016873]
  Current debiasing approaches often result in a degradation in model capabilities such as factual accuracy and knowledge retention.<n>We propose a contrastive learning framework that learns through carefully constructed positive and negative examples.
  arXiv  Detail & Related papers  (2025-05-25T21:26:18Z)
• Mitigate One, Skew Another? Tackling Intersectional Biases in Text-to-Image Models [73.20190633746442]
  We introduce BiasConnect, a novel tool for analyzing and quantifying bias interactions in text-to-image models.<n>We propose InterMit, an intersectional bias mitigation algorithm guided by user-defined target distributions and priority weights.
  arXiv  Detail & Related papers  (2025-05-22T20:56:38Z)
• Evaluate Bias without Manual Test Sets: A Concept Representation Perspective for LLMs [25.62533031580287]
  Bias in Large Language Models (LLMs) significantly undermines their reliability and fairness.<n>We propose BiasLens, a test-set-free bias analysis framework based on the structure of the model's vector space.
  arXiv  Detail & Related papers  (2025-05-21T13:50:23Z)
• Fairness Mediator: Neutralize Stereotype Associations to Mitigate Bias in Large Language Models [66.5536396328527]
  LLMs inadvertently absorb spurious correlations from training data, leading to stereotype associations between biased concepts and specific social groups.<n>We propose Fairness Mediator (FairMed), a bias mitigation framework that neutralizes stereotype associations.<n>Our framework comprises two main components: a stereotype association prober and an adversarial debiasing neutralizer.
  arXiv  Detail & Related papers  (2025-04-10T14:23:06Z)
• DebCSE: Rethinking Unsupervised Contrastive Sentence Embedding Learning in the Debiasing Perspective [1.351603931922027]
  We argue that effectively eliminating the influence of various biases is crucial for learning high-quality sentence embeddings. We propose a novel contrastive framework for sentence embedding, termed DebCSE, which can eliminate the impact of these biases.
  arXiv  Detail & Related papers  (2023-09-14T02:43:34Z)
• Provable Unrestricted Adversarial Training without Compromise with Generalizability [44.02361569894942]
  Adversarial training (AT) is widely considered as the most promising strategy to defend against adversarial attacks. The existing AT methods often achieve adversarial robustness at the expense of standard generalizability. We propose a novel AT approach called Provable Unrestricted Adversarial Training (PUAT)
  arXiv  Detail & Related papers  (2023-01-22T07:45:51Z)
• Ensembling over Classifiers: a Bias-Variance Perspective [13.006468721874372]
  We build upon the extension to the bias-variance decomposition by Pfau (2013) in order to gain crucial insights into the behavior of ensembles of classifiers. We show that conditional estimates necessarily incur an irreducible error. Empirically, standard ensembling reducesthe bias, leading us to hypothesize that ensembles of classifiers may perform well in part because of this unexpected reduction.
  arXiv  Detail & Related papers  (2022-06-21T17:46:35Z)
• Debiased Contrastive Learning of Unsupervised Sentence Representations [88.58117410398759]
  Contrastive learning is effective in improving pre-trained language models (PLM) to derive high-quality sentence representations. Previous works mostly adopt in-batch negatives or sample from training data at random. We present a new framework textbfDCLR to alleviate the influence of these improper negatives.
  arXiv  Detail & Related papers  (2022-05-02T05:07:43Z)
• PCL: Peer-Contrastive Learning with Diverse Augmentations for Unsupervised Sentence Embeddings [69.87899694963251]
  We propose a novel Peer-Contrastive Learning (PCL) with diverse augmentations. PCL constructs diverse contrastive positives and negatives at the group level for unsupervised sentence embeddings. PCL can perform peer-positive contrast as well as peer-network cooperation, which offers an inherent anti-bias ability.
  arXiv  Detail & Related papers  (2022-01-28T13:02:41Z)
• Unsupervised Learning of Debiased Representations with Pseudo-Attributes [85.5691102676175]
  We propose a simple but effective debiasing technique in an unsupervised manner. We perform clustering on the feature embedding space and identify pseudoattributes by taking advantage of the clustering results. We then employ a novel cluster-based reweighting scheme for learning debiased representation.
  arXiv  Detail & Related papers  (2021-08-06T05:20:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.