Decorrelate Irrelevant, Purify Relevant: Overcome Textual Spurious Correlations from a Feature Perspective

• URL: http://arxiv.org/abs/2202.08048v1
• Date: Wed, 16 Feb 2022 13:23:14 GMT
• Title: Decorrelate Irrelevant, Purify Relevant: Overcome Textual Spurious Correlations from a Feature Perspective
• Authors: Shihan Dou, Rui Zheng, Ting Wu, Songyang Gao, Qi Zhang, Yueming Wu, Xuanjing Huang
• Abstract summary: Natural language understanding (NLU) models tend to rely on spurious correlations (emphi.e., dataset bias) to achieve high performance on in-distribution datasets but poor performance on out-of-distribution ones. Most of the existing debiasing methods often identify and weaken these samples with biased features. Down-weighting these samples obstructs the model in learning from the non-biased parts of these samples. We propose to eliminate spurious correlations in a fine-grained manner from a feature space perspective.
• Score: 47.10907370311025
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Natural language understanding (NLU) models tend to rely on spurious correlations (\emph{i.e.}, dataset bias) to achieve high performance on in-distribution datasets but poor performance on out-of-distribution ones. Most of the existing debiasing methods often identify and weaken these samples with biased features (\emph{i.e.}, superficial surface features that cause such spurious correlations). However, down-weighting these samples obstructs the model in learning from the non-biased parts of these samples. To tackle this challenge, in this paper, we propose to eliminate spurious correlations in a fine-grained manner from a feature space perspective. Specifically, we introduce Random Fourier Features and weighted re-sampling to decorrelate the dependencies between features to mitigate spurious correlations. After obtaining decorrelated features, we further design a mutual-information-based method to purify them, which forces the model to learn features that are more relevant to tasks. Extensive experiments on two well-studied NLU tasks including Natural Language Inference and Fact Verification demonstrate that our method is superior to other comparative approaches.

Related papers

• Towards Robust Text Classification: Mitigating Spurious Correlations with Causal Learning [2.7813683000222653]
  We propose the Causally Calibrated Robust ( CCR) to reduce models' reliance on spurious correlations. CCR integrates a causal feature selection method based on counterfactual reasoning, along with an inverse propensity weighting (IPW) loss function. We show that CCR state-of-the-art performance among methods without group labels, and in some cases, it can compete with the models that utilize group labels.
  arXiv  Detail & Related papers  (2024-11-01T21:29:07Z)
• A Simple Remedy for Dataset Bias via Self-Influence: A Mislabeled Sample Perspective [33.78421391776591]
  In this paper, we propose a novel perspective of mislabeled sample detection. We show that our new perspective can boost the precision of detection and rectify biased models effectively. Our approach is complementary to existing methods, showing performance improvement even when applied to models that have already undergone recent debiasing techniques.
  arXiv  Detail & Related papers  (2024-11-01T04:54:32Z)
• Causal Feature Selection via Transfer Entropy [59.999594949050596]
  Causal discovery aims to identify causal relationships between features with observational data. We introduce a new causal feature selection approach that relies on the forward and backward feature selection procedures. We provide theoretical guarantees on the regression and classification errors for both the exact and the finite-sample cases.
  arXiv  Detail & Related papers  (2023-10-17T08:04:45Z)
• FACTS: First Amplify Correlations and Then Slice to Discover Bias [17.244153084361102]
  Computer vision datasets frequently contain spurious correlations between task-relevant labels and (easy to learn) latent task-irrelevant attributes. Models trained on such datasets learn "shortcuts" and underperform on bias-conflicting slices of data where the correlation does not hold. We propose First Amplify Correlations and Then Slice to Discover Bias to inform downstream bias mitigation strategies.
  arXiv  Detail & Related papers  (2023-09-29T17:41:26Z)
• Stubborn Lexical Bias in Data and Models [50.79738900885665]
  We use a new statistical method to examine whether spurious patterns in data appear in models trained on the data. We apply an optimization approach to *reweight* the training data, reducing thousands of spurious correlations. Surprisingly, though this method can successfully reduce lexical biases in the training data, we still find strong evidence of corresponding bias in the trained models.
  arXiv  Detail & Related papers  (2023-06-03T20:12:27Z)
• Boosting Differentiable Causal Discovery via Adaptive Sample Reweighting [62.23057729112182]
  Differentiable score-based causal discovery methods learn a directed acyclic graph from observational data. We propose a model-agnostic framework to boost causal discovery performance by dynamically learning the adaptive weights for the Reweighted Score function, ReScore.
  arXiv  Detail & Related papers  (2023-03-06T14:49:59Z)
• Feature-Level Debiased Natural Language Understanding [86.8751772146264]
  Existing natural language understanding (NLU) models often rely on dataset biases to achieve high performance on specific datasets. We propose debiasing contrastive learning (DCT) to mitigate biased latent features and neglect the dynamic nature of bias. DCT outperforms state-of-the-art baselines on out-of-distribution datasets while maintaining in-distribution performance.
  arXiv  Detail & Related papers  (2022-12-11T06:16:14Z)
• Breaking the Spurious Causality of Conditional Generation via Fairness Intervention with Corrective Sampling [77.15766509677348]
  Conditional generative models often inherit spurious correlations from the training dataset. This can result in label-conditional distributions that are imbalanced with respect to another latent attribute. We propose a general two-step strategy to mitigate this issue.
  arXiv  Detail & Related papers  (2022-12-05T08:09:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.