ProtoFair: Fair Self-Supervised Contrastive Learning via Pseudo-Counterfactual Pairs

• URL: http://arxiv.org/abs/2605.01971v1
• Date: Sun, 03 May 2026 17:07:54 GMT
• Title: ProtoFair: Fair Self-Supervised Contrastive Learning via Pseudo-Counterfactual Pairs
• Abstract summary: Existing fairness-aware methods address this by redesigning the self-supervised objective itself.<n>We propose ProtoFair, a fairness-aware contrastive loss designed to work alongside existing SSL objectives without modifying them.<n>ProtoFair integrates seamlessly with both SimCLR and SupCon.
• Score: 5.192293017737767
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Self-supervised learning methods learn high-quality visual representations, yet recent studies show that these representations often capture demographic biases present in the training data. Existing fairness-aware methods address this by redesigning the self-supervised objective itself, limiting portability across the rapidly evolving landscape of self-supervised learning (SSL) frameworks. We propose ProtoFair, a fairness-aware contrastive loss designed to work alongside existing SSL objectives without modifying them. ProtoFair leverages unsupervised prototype clustering to identify pseudo-counterfactual pairs: samples sharing the same cluster assignment but belonging to different sensitive groups. By pulling these content-matched, cross-group samples together in the embedding space, ProtoFair encourages the encoder to learn representations that are invariant to the sensitive attribute. The method requires only sensitive attribute annotations, no target labels, and integrates seamlessly with both SimCLR and SupCon. Experiments on CelebA and UTKFace demonstrate consistent fairness improvements while maintaining competitive accuracy.

Related papers

• A Probabilistic Model Behind Self-Supervised Learning [53.64989127914936]
  In self-supervised learning (SSL), representations are learned via an auxiliary task without annotated labels. We present a generative latent variable model for self-supervised learning. We show that several families of discriminative SSL, including contrastive methods, induce a comparable distribution over representations.
  arXiv  Detail & Related papers  (2024-02-02T13:31:17Z)
• DualFair: Fair Representation Learning at Both Group and Individual Levels via Contrastive Self-supervision [73.80009454050858]
  This work presents a self-supervised model, called DualFair, that can debias sensitive attributes like gender and race from learned representations. Our model jointly optimize for two fairness criteria - group fairness and counterfactual fairness.
  arXiv  Detail & Related papers  (2023-03-15T07:13:54Z)
• Beyond Instance Discrimination: Relation-aware Contrastive Self-supervised Learning [75.46664770669949]
  We present relation-aware contrastive self-supervised learning (ReCo) to integrate instance relations. Our ReCo consistently gains remarkable performance improvements.
  arXiv  Detail & Related papers  (2022-11-02T03:25:28Z)
• On Higher Adversarial Susceptibility of Contrastive Self-Supervised Learning [104.00264962878956]
  Contrastive self-supervised learning (CSL) has managed to match or surpass the performance of supervised learning in image and video classification. It is still largely unknown if the nature of the representation induced by the two learning paradigms is similar. We identify the uniform distribution of data representation over a unit hypersphere in the CSL representation space as the key contributor to this phenomenon. We devise strategies that are simple, yet effective in improving model robustness with CSL training.
  arXiv  Detail & Related papers  (2022-07-22T03:49:50Z)
• Learning Fair Representation via Distributional Contrastive Disentanglement [9.577369164287813]
  Learning fair representation is crucial for achieving fairness or debiasing sensitive information. We propose a new approach, learning FAir Representation via distributional CONtrastive Variational AutoEncoder (FarconVAE) We show superior performance on fairness, pretrained model debiasing, and domain generalization tasks from various modalities.
  arXiv  Detail & Related papers  (2022-06-17T12:58:58Z)
• Fair Contrastive Learning for Facial Attribute Classification [25.436462696033846]
  We propose a new Fair Supervised Contrastive Loss (FSCL) for fair visual representation learning. In this paper, we for the first time analyze unfairness caused by supervised contrastive learning. Our method is robust to the intensity of data bias and effectively works in incomplete supervised settings.
  arXiv  Detail & Related papers  (2022-03-30T11:16:18Z)
• Dual Contrastive Learning: Text Classification via Label-Aware Data Augmentation [38.61781713469419]
  We introduce a dual contrastive learning framework that simultaneously learns the features of input samples and the parameters of classifiers in the same space. Specifically, DualCL regards the parameters of the classifiers as augmented samples associating to different labels and then exploits the contrastive learning between the input samples and the augmented samples. Empirical studies on five benchmark text classification datasets demonstrate the improvement in classification accuracy and confirm the capability of learning discriminative representations of DualCL.
  arXiv  Detail & Related papers  (2022-01-21T13:59:45Z)
• Latent Space Smoothing for Individually Fair Representations [12.739528232133495]
  We introduce LASSI, the first representation learning method for certifying individual fairness of high-dimensional data. Our key insight is to leverage recent advances in generative modeling to capture the set of similar individuals in the generative latent space. We employ randomized smoothing to provably map similar individuals close together, in turn ensuring that local robustness verification of the downstream application results in end-to-end fairness certification.
  arXiv  Detail & Related papers  (2021-11-26T18:22:42Z)
• Contrastive Learning for Fair Representations [50.95604482330149]
  Trained classification models can unintentionally lead to biased representations and predictions. Existing debiasing methods for classification models, such as adversarial training, are often expensive to train and difficult to optimise. We propose a method for mitigating bias by incorporating contrastive learning, in which instances sharing the same class label are encouraged to have similar representations.
  arXiv  Detail & Related papers  (2021-09-22T10:47:51Z)
• Neighborhood Contrastive Learning for Novel Class Discovery [79.14767688903028]
  We build a new framework, named Neighborhood Contrastive Learning, to learn discriminative representations that are important to clustering performance. We experimentally demonstrate that these two ingredients significantly contribute to clustering performance and lead our model to outperform state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2021-06-20T17:34:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.