Latent Adversarial Debiasing: Mitigating Collider Bias in Deep Neural
Networks
- URL: http://arxiv.org/abs/2011.11486v1
- Date: Thu, 19 Nov 2020 10:53:45 GMT
- Title: Latent Adversarial Debiasing: Mitigating Collider Bias in Deep Neural
Networks
- Authors: Luke Darlow, Stanis{\l}aw Jastrz\k{e}bski, Amos Storkey
- Abstract summary: Collider bias is a harmful form of sample selection bias that neural networks are ill-equipped to handle.
We show it is possible to mitigate against this by generating bias-decoupled training data using latent adversarial debiasing.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Collider bias is a harmful form of sample selection bias that neural networks
are ill-equipped to handle. This bias manifests itself when the underlying
causal signal is strongly correlated with other confounding signals due to the
training data collection procedure. In the situation where the confounding
signal is easy-to-learn, deep neural networks will latch onto this and the
resulting model will generalise poorly to in-the-wild test scenarios. We argue
herein that the cause of failure is a combination of the deep structure of
neural networks and the greedy gradient-driven learning process used - one that
prefers easy-to-compute signals when available. We show it is possible to
mitigate against this by generating bias-decoupled training data using latent
adversarial debiasing (LAD), even when the confounding signal is present in
100% of the training data. By training neural networks on these adversarial
examples,we can improve their generalisation in collider bias settings.
Experiments show state-of-the-art performance of LAD in label-free debiasing
with gains of 76.12% on background coloured MNIST, 35.47% on fore-ground
coloured MNIST, and 8.27% on corrupted CIFAR-10.
Related papers
- Deep Neural Networks Tend To Extrapolate Predictably [51.303814412294514]
neural network predictions tend to be unpredictable and overconfident when faced with out-of-distribution (OOD) inputs.
We observe that neural network predictions often tend towards a constant value as input data becomes increasingly OOD.
We show how one can leverage our insights in practice to enable risk-sensitive decision-making in the presence of OOD inputs.
arXiv Detail & Related papers (2023-10-02T03:25:32Z) - Signal Is Harder To Learn Than Bias: Debiasing with Focal Loss [10.031357641396616]
neural networks are notorious for learning unwanted associations, also known as biases, instead of the underlying decision rule.
We propose Signal is Harder, a variational-autoencoder-based method that simultaneously trains a biased and unbiased classifier.
We propose a perturbation scheme in the latent space for visualizing the bias that helps practitioners become aware of the sources of spurious correlations.
arXiv Detail & Related papers (2023-05-31T09:09:59Z) - Benign Overfitting for Two-layer ReLU Convolutional Neural Networks [60.19739010031304]
We establish algorithm-dependent risk bounds for learning two-layer ReLU convolutional neural networks with label-flipping noise.
We show that, under mild conditions, the neural network trained by gradient descent can achieve near-zero training loss and Bayes optimal test risk.
arXiv Detail & Related papers (2023-03-07T18:59:38Z) - Neural networks trained with SGD learn distributions of increasing
complexity [78.30235086565388]
We show that neural networks trained using gradient descent initially classify their inputs using lower-order input statistics.
We then exploit higher-order statistics only later during training.
We discuss the relation of DSB to other simplicity biases and consider its implications for the principle of universality in learning.
arXiv Detail & Related papers (2022-11-21T15:27:22Z) - Self-supervised debiasing using low rank regularization [59.84695042540525]
Spurious correlations can cause strong biases in deep neural networks, impairing generalization ability.
We propose a self-supervised debiasing framework potentially compatible with unlabeled samples.
Remarkably, the proposed debiasing framework significantly improves the generalization performance of self-supervised learning baselines.
arXiv Detail & Related papers (2022-10-11T08:26:19Z) - Compensating trajectory bias for unsupervised patient stratification
using adversarial recurrent neural networks [0.6323908398583082]
We show that patient embeddings and clusters might be impacted by a trajectory bias.
Results are dominated by the amount of data contained in each patients trajectory, instead of clinically relevant details.
We present a method that can overcome this issue using an adversarial training scheme on top of a RNN-AE.
arXiv Detail & Related papers (2021-12-14T09:01:28Z) - Towards an Understanding of Benign Overfitting in Neural Networks [104.2956323934544]
Modern machine learning models often employ a huge number of parameters and are typically optimized to have zero training loss.
We examine how these benign overfitting phenomena occur in a two-layer neural network setting.
We show that it is possible for the two-layer ReLU network interpolator to achieve a near minimax-optimal learning rate.
arXiv Detail & Related papers (2021-06-06T19:08:53Z) - Learning from Failure: Training Debiased Classifier from Biased
Classifier [76.52804102765931]
We show that neural networks learn to rely on spurious correlation only when it is "easier" to learn than the desired knowledge.
We propose a failure-based debiasing scheme by training a pair of neural networks simultaneously.
Our method significantly improves the training of the network against various types of biases in both synthetic and real-world datasets.
arXiv Detail & Related papers (2020-07-06T07:20:29Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.