Directional Bias Amplification
- URL: http://arxiv.org/abs/2102.12594v1
- Date: Wed, 24 Feb 2021 22:54:21 GMT
- Title: Directional Bias Amplification
- Authors: Angelina Wang and Olga Russakovsky
- Abstract summary: Bias amplification is the tendency of models to amplify the biases present in the data they are trained on.
A metric for measuring bias amplification was introduced in the seminal work by Zhao et al.
We introduce and analyze a new, decoupled metric for measuring bias amplification, $textBiasAmp_rightarrow$ (Directional Bias Amplification)
- Score: 21.482317675176443
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Mitigating bias in machine learning systems requires refining our
understanding of bias propagation pathways: from societal structures to
large-scale data to trained models to impact on society. In this work, we focus
on one aspect of the problem, namely bias amplification: the tendency of models
to amplify the biases present in the data they are trained on. A metric for
measuring bias amplification was introduced in the seminal work by Zhao et al.
(2017); however, as we demonstrate, this metric suffers from a number of
shortcomings including conflating different types of bias amplification and
failing to account for varying base rates of protected classes. We introduce
and analyze a new, decoupled metric for measuring bias amplification,
$\text{BiasAmp}_{\rightarrow}$ (Directional Bias Amplification). We thoroughly
analyze and discuss both the technical assumptions and the normative
implications of this metric. We provide suggestions about its measurement by
cautioning against predicting sensitive attributes, encouraging the use of
confidence intervals due to fluctuations in the fairness of models across runs,
and discussing the limitations of what this metric captures. Throughout this
paper, we work to provide an interrogative look at the technical measurement of
bias amplification, guided by our normative ideas of what we want it to
encompass.
Related papers
- Take Care of Your Prompt Bias! Investigating and Mitigating Prompt Bias in Factual Knowledge Extraction [56.17020601803071]
Recent research shows that pre-trained language models (PLMs) suffer from "prompt bias" in factual knowledge extraction.
This paper aims to improve the reliability of existing benchmarks by thoroughly investigating and mitigating prompt bias.
arXiv Detail & Related papers (2024-03-15T02:04:35Z) - Men Also Do Laundry: Multi-Attribute Bias Amplification [2.492300648514129]
Computer vision systems are not only reproducing but amplifying harmful social biases.
We propose a new metric: Multi-Attribute Bias Amplification.
We validate our proposed metric through an analysis of gender bias amplification on the COCO and imSitu datasets.
arXiv Detail & Related papers (2022-10-21T12:50:15Z) - Self-supervised debiasing using low rank regularization [59.84695042540525]
Spurious correlations can cause strong biases in deep neural networks, impairing generalization ability.
We propose a self-supervised debiasing framework potentially compatible with unlabeled samples.
Remarkably, the proposed debiasing framework significantly improves the generalization performance of self-supervised learning baselines.
arXiv Detail & Related papers (2022-10-11T08:26:19Z) - Prisoners of Their Own Devices: How Models Induce Data Bias in
Performative Prediction [4.874780144224057]
A biased model can make decisions that disproportionately harm certain groups in society.
Much work has been devoted to measuring unfairness in static ML environments, but not in dynamic, performative prediction ones.
We propose a taxonomy to characterize bias in the data, and study cases where it is shaped by model behaviour.
arXiv Detail & Related papers (2022-06-27T10:56:04Z) - A Systematic Study of Bias Amplification [16.245943270343343]
Recent research suggests that predictions made by machine-learning models can amplify biases present in the training data.
We perform the first systematic, controlled study into when and how bias amplification occurs.
arXiv Detail & Related papers (2022-01-27T18:04:24Z) - Information-Theoretic Bias Reduction via Causal View of Spurious
Correlation [71.9123886505321]
We propose an information-theoretic bias measurement technique through a causal interpretation of spurious correlation.
We present a novel debiasing framework against the algorithmic bias, which incorporates a bias regularization loss.
The proposed bias measurement and debiasing approaches are validated in diverse realistic scenarios.
arXiv Detail & Related papers (2022-01-10T01:19:31Z) - Balancing out Bias: Achieving Fairness Through Training Reweighting [58.201275105195485]
Bias in natural language processing arises from models learning characteristics of the author such as gender and race.
Existing methods for mitigating and measuring bias do not directly account for correlations between author demographics and linguistic variables.
This paper introduces a very simple but highly effective method for countering bias using instance reweighting.
arXiv Detail & Related papers (2021-09-16T23:40:28Z) - Improving Robustness by Augmenting Training Sentences with
Predicate-Argument Structures [62.562760228942054]
Existing approaches to improve robustness against dataset biases mostly focus on changing the training objective.
We propose to augment the input sentences in the training data with their corresponding predicate-argument structures.
We show that without targeting a specific bias, our sentence augmentation improves the robustness of transformer models against multiple biases.
arXiv Detail & Related papers (2020-10-23T16:22:05Z) - Mitigating Gender Bias Amplification in Distribution by Posterior
Regularization [75.3529537096899]
We investigate the gender bias amplification issue from the distribution perspective.
We propose a bias mitigation approach based on posterior regularization.
Our study sheds the light on understanding the bias amplification.
arXiv Detail & Related papers (2020-05-13T11:07:10Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.