A Systematic Study of Bias Amplification
- URL: http://arxiv.org/abs/2201.11706v1
- Date: Thu, 27 Jan 2022 18:04:24 GMT
- Title: A Systematic Study of Bias Amplification
- Authors: Melissa Hall, Laurens van der Maaten, Laura Gustafson, Aaron Adcock
- Abstract summary: Recent research suggests that predictions made by machine-learning models can amplify biases present in the training data.
We perform the first systematic, controlled study into when and how bias amplification occurs.
- Score: 16.245943270343343
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Recent research suggests that predictions made by machine-learning models can
amplify biases present in the training data. When a model amplifies bias, it
makes certain predictions at a higher rate for some groups than expected based
on training-data statistics. Mitigating such bias amplification requires a deep
understanding of the mechanics in modern machine learning that give rise to
that amplification. We perform the first systematic, controlled study into when
and how bias amplification occurs. To enable this study, we design a simple
image-classification problem in which we can tightly control (synthetic)
biases. Our study of this problem reveals that the strength of bias
amplification is correlated to measures such as model accuracy, model capacity,
model overconfidence, and amount of training data. We also find that bias
amplification can vary greatly during training. Finally, we find that bias
amplification may depend on the difficulty of the classification task relative
to the difficulty of recognizing group membership: bias amplification appears
to occur primarily when it is easier to recognize group membership than class
membership. Our results suggest best practices for training machine-learning
models that we hope will help pave the way for the development of better
mitigation strategies.
Related papers
- An Effective Theory of Bias Amplification [18.648588509429167]
Machine learning models may capture and amplify biases present in data, leading to disparate test performance across social groups.
We propose a precise analytical theory in the context of ridge regression, where the former models neural networks in a simplified regime.
Our theory offers a unified and rigorous explanation of machine learning bias, providing insights into phenomena such as bias amplification and minority-group bias.
arXiv Detail & Related papers (2024-10-07T08:43:22Z) - On the Inductive Bias of Stacking Towards Improving Reasoning [50.225873619537765]
We propose a variant of gradual stacking called MIDAS that can speed up language model training by up to 40%.
MIDAS is not only training-efficient but surprisingly also has an inductive bias towards improving downstream tasks.
We conjecture the underlying reason for this inductive bias by exploring the connection of stacking to looped models.
arXiv Detail & Related papers (2024-09-27T17:58:21Z) - Model Debiasing by Learnable Data Augmentation [19.625915578646758]
This paper proposes a novel 2-stage learning pipeline featuring a data augmentation strategy able to regularize the training.
Experiments on synthetic and realistic biased datasets show state-of-the-art classification accuracy, outperforming competing methods.
arXiv Detail & Related papers (2024-08-09T09:19:59Z) - Fast Model Debias with Machine Unlearning [54.32026474971696]
Deep neural networks might behave in a biased manner in many real-world scenarios.
Existing debiasing methods suffer from high costs in bias labeling or model re-training.
We propose a fast model debiasing framework (FMD) which offers an efficient approach to identify, evaluate and remove biases.
arXiv Detail & Related papers (2023-10-19T08:10:57Z) - CMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep
Learning [55.733193075728096]
Modern deep neural networks can easily overfit to biased training data containing corrupted labels or class imbalance.
Sample re-weighting methods are popularly used to alleviate this data bias issue.
We propose a meta-model capable of adaptively learning an explicit weighting scheme directly from data.
arXiv Detail & Related papers (2022-02-11T13:49:51Z) - Evading the Simplicity Bias: Training a Diverse Set of Models Discovers
Solutions with Superior OOD Generalization [93.8373619657239]
Neural networks trained with SGD were recently shown to rely preferentially on linearly-predictive features.
This simplicity bias can explain their lack of robustness out of distribution (OOD)
We demonstrate that the simplicity bias can be mitigated and OOD generalization improved.
arXiv Detail & Related papers (2021-05-12T12:12:24Z) - Directional Bias Amplification [21.482317675176443]
Bias amplification is the tendency of models to amplify the biases present in the data they are trained on.
A metric for measuring bias amplification was introduced in the seminal work by Zhao et al.
We introduce and analyze a new, decoupled metric for measuring bias amplification, $textBiasAmp_rightarrow$ (Directional Bias Amplification)
arXiv Detail & Related papers (2021-02-24T22:54:21Z) - Learning from Failure: Training Debiased Classifier from Biased
Classifier [76.52804102765931]
We show that neural networks learn to rely on spurious correlation only when it is "easier" to learn than the desired knowledge.
We propose a failure-based debiasing scheme by training a pair of neural networks simultaneously.
Our method significantly improves the training of the network against various types of biases in both synthetic and real-world datasets.
arXiv Detail & Related papers (2020-07-06T07:20:29Z) - Mitigating Gender Bias Amplification in Distribution by Posterior
Regularization [75.3529537096899]
We investigate the gender bias amplification issue from the distribution perspective.
We propose a bias mitigation approach based on posterior regularization.
Our study sheds the light on understanding the bias amplification.
arXiv Detail & Related papers (2020-05-13T11:07:10Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.