Contrastive Learning for Fair Representations
- URL: http://arxiv.org/abs/2109.10645v1
- Date: Wed, 22 Sep 2021 10:47:51 GMT
- Title: Contrastive Learning for Fair Representations
- Authors: Aili Shen, Xudong Han, Trevor Cohn, Timothy Baldwin, Lea Frermann
- Abstract summary: Trained classification models can unintentionally lead to biased representations and predictions.
Existing debiasing methods for classification models, such as adversarial training, are often expensive to train and difficult to optimise.
We propose a method for mitigating bias by incorporating contrastive learning, in which instances sharing the same class label are encouraged to have similar representations.
- Score: 50.95604482330149
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Trained classification models can unintentionally lead to biased
representations and predictions, which can reinforce societal preconceptions
and stereotypes. Existing debiasing methods for classification models, such as
adversarial training, are often expensive to train and difficult to optimise.
In this paper, we propose a method for mitigating bias in classifier training
by incorporating contrastive learning, in which instances sharing the same
class label are encouraged to have similar representations, while instances
sharing a protected attribute are forced further apart. In such a way our
method learns representations which capture the task label in focused regions,
while ensuring the protected attribute has diverse spread, and thus has limited
impact on prediction and thereby results in fairer models. Extensive
experimental results across four tasks in NLP and computer vision show (a) that
our proposed method can achieve fairer representations and realises bias
reductions compared with competitive baselines; and (b) that it can do so
without sacrificing main task performance; (c) that it sets a new
state-of-the-art performance in one task despite reducing the bias. Finally,
our method is conceptually simple and agnostic to network architectures, and
incurs minimal additional compute cost.
Related papers
- The Bad Batches: Enhancing Self-Supervised Learning in Image Classification Through Representative Batch Curation [1.519321208145928]
The pursuit of learning robust representations without human supervision is a longstanding challenge.
This paper attempts to alleviate the influence of false positive and false negative pairs by employing pairwise similarity calculations through the Fr'echet ResNet Distance (FRD)
The effectiveness of the proposed method is substantiated by empirical results, where a linear classifier trained on self-supervised contrastive representations achieved an impressive 87.74% top-1 accuracy on STL10 and 99.31% on the Flower102 dataset.
arXiv Detail & Related papers (2024-03-28T17:04:07Z) - A Probabilistic Model Behind Self-Supervised Learning [53.64989127914936]
In self-supervised learning (SSL), representations are learned via an auxiliary task without annotated labels.
We present a generative latent variable model for self-supervised learning.
We show that several families of discriminative SSL, including contrastive methods, induce a comparable distribution over representations.
arXiv Detail & Related papers (2024-02-02T13:31:17Z) - Optimising Equal Opportunity Fairness in Model Training [60.0947291284978]
Existing debiasing methods, such as adversarial training and removing protected information from representations, have been shown to reduce bias.
We propose two novel training objectives which directly optimise for the widely-used criterion of it equal opportunity, and show that they are effective in reducing bias while maintaining high performance over two classification tasks.
arXiv Detail & Related papers (2022-05-05T01:57:58Z) - Fair Contrastive Learning for Facial Attribute Classification [25.436462696033846]
We propose a new Fair Supervised Contrastive Loss (FSCL) for fair visual representation learning.
In this paper, we for the first time analyze unfairness caused by supervised contrastive learning.
Our method is robust to the intensity of data bias and effectively works in incomplete supervised settings.
arXiv Detail & Related papers (2022-03-30T11:16:18Z) - A Low Rank Promoting Prior for Unsupervised Contrastive Learning [108.91406719395417]
We construct a novel probabilistic graphical model that effectively incorporates the low rank promoting prior into the framework of contrastive learning.
Our hypothesis explicitly requires that all the samples belonging to the same instance class lie on the same subspace with small dimension.
Empirical evidences show that the proposed algorithm clearly surpasses the state-of-the-art approaches on multiple benchmarks.
arXiv Detail & Related papers (2021-08-05T15:58:25Z) - Semi-Supervised Few-Shot Classification with Deep Invertible Hybrid
Models [4.189643331553922]
We propose a deep invertible hybrid model which integrates discriminative and generative learning at a latent space level for semi-supervised few-shot classification.
Our main originality lies in our integration of these components at a latent space level, which is effective in preventing overfitting.
arXiv Detail & Related papers (2021-05-22T05:55:16Z) - An Effective Baseline for Robustness to Distributional Shift [5.627346969563955]
Refraining from confidently predicting when faced with categories of inputs different from those seen during training is an important requirement for the safe deployment of deep learning systems.
We present a simple, but highly effective approach to deal with out-of-distribution detection that uses the principle of abstention.
arXiv Detail & Related papers (2021-05-15T00:46:11Z) - Unleashing the Power of Contrastive Self-Supervised Visual Models via
Contrast-Regularized Fine-Tuning [94.35586521144117]
We investigate whether applying contrastive learning to fine-tuning would bring further benefits.
We propose Contrast-regularized tuning (Core-tuning), a novel approach for fine-tuning contrastive self-supervised visual models.
arXiv Detail & Related papers (2021-02-12T16:31:24Z) - Learning Diverse Representations for Fast Adaptation to Distribution
Shift [78.83747601814669]
We present a method for learning multiple models, incorporating an objective that pressures each to learn a distinct way to solve the task.
We demonstrate our framework's ability to facilitate rapid adaptation to distribution shift.
arXiv Detail & Related papers (2020-06-12T12:23:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.