Mitigating Spurious Correlations with Memorization-Guided Dataset De-Biasing

• URL: http://arxiv.org/abs/2606.02830v1
• Date: Mon, 01 Jun 2026 19:48:20 GMT
• Title: Mitigating Spurious Correlations with Memorization-Guided Dataset De-Biasing
• Abstract summary: We develop a two-stage sample scoring function that disentangles the learning dynamics of core and spurious features.<n>Based on our proposed metric, we introduce a new algorithm to find and prioritize informative samples both with and without spurious correlations.
• Score: 8.645414910735449
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Real-world datasets often contain spurious correlations that are not causally related to the target label. When such correlations dominate the majority of training samples, models tend to rely on them, leading to misclassification of minority samples that do not exhibit the same spurious patterns. While a potential approach is to select subsets of data to better represent the minority samples, this may require access to group labels, which are typically unknown. Furthermore, as we demonstrate, widely used sample scoring functions in the invariant subset or coreset selection literature largely depend on spurious features and therefore fail to accurately capture the importance or difficulty of core, causally relevant features. Accordingly, we propose to mitigate spurious correlations by developing a two-stage sample scoring function that disentangles the learning dynamics of core and spurious features and evaluates their difficulty separately. Based on our proposed metric, we introduce a new algorithm to find and prioritize informative samples both with and without spurious correlations. Extensive experiments demonstrate that a standard ERM model trained on our selected samples achieves superior performance compared to state-of-the-art debiasing techniques, while requiring as little as 10\% of the original training data.

Related papers

• Let Samples Speak: Mitigating Spurious Correlation by Exploiting the Clusterness of Samples [11.727747752958436]
  Deep learning models often learn features that spuriously correlate with the class label during training but are irrelevant to the prediction task.<n>Existing methods typically address this issue by annotating potential spurious attributes, or filtering spurious features based on some empirical assumptions.<n>We propose a data-oriented approach to mitigate the spurious correlation in deep learning models.
  arXiv  Detail & Related papers  (2025-12-28T10:54:51Z)
• The Impact of Coreset Selection on Spurious Correlations and Group Robustness [27.47900711644272]
  Coreset selection methods have shown promise in reducing the training data size while maintaining model performance for data-efficient machine learning.<n>We conduct the first comprehensive analysis of the implications of data selection on the spurious bias levels of the selected coresets and the robustness of downstream models trained on them.
  arXiv  Detail & Related papers  (2025-07-15T19:46:30Z)
• DDB: Diffusion Driven Balancing to Address Spurious Correlations [24.940576844328408]
  Deep neural networks trained with Empirical Risk Minimization often fail to generalize to out-of-distribution samples.<n>We propose a Diffusion Driven Balancing (DDB) technique to generate training samples with text-to-image diffusion models.<n>Our experiments show that our technique achieves better worst-group accuracy than the existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-03-21T15:28:22Z)
• A Simple Remedy for Dataset Bias via Self-Influence: A Mislabeled Sample Perspective [33.78421391776591]
  In this paper, we propose a novel perspective of mislabeled sample detection. We show that our new perspective can boost the precision of detection and rectify biased models effectively. Our approach is complementary to existing methods, showing performance improvement even when applied to models that have already undergone recent debiasing techniques.
  arXiv  Detail & Related papers  (2024-11-01T04:54:32Z)
• Downstream-Pretext Domain Knowledge Traceback for Active Learning [138.02530777915362]
  We propose a downstream-pretext domain knowledge traceback (DOKT) method that traces the data interactions of downstream knowledge and pre-training guidance. DOKT consists of a traceback diversity indicator and a domain-based uncertainty estimator. Experiments conducted on ten datasets show that our model outperforms other state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-07-20T01:34:13Z)
• Tackling Diverse Minorities in Imbalanced Classification [80.78227787608714]
  Imbalanced datasets are commonly observed in various real-world applications, presenting significant challenges in training classifiers. We propose generating synthetic samples iteratively by mixing data samples from both minority and majority classes. We demonstrate the effectiveness of our proposed framework through extensive experiments conducted on seven publicly available benchmark datasets.
  arXiv  Detail & Related papers  (2023-08-28T18:48:34Z)
• Stubborn Lexical Bias in Data and Models [50.79738900885665]
  We use a new statistical method to examine whether spurious patterns in data appear in models trained on the data. We apply an optimization approach to *reweight* the training data, reducing thousands of spurious correlations. Surprisingly, though this method can successfully reduce lexical biases in the training data, we still find strong evidence of corresponding bias in the trained models.
  arXiv  Detail & Related papers  (2023-06-03T20:12:27Z)
• Self-Evolution Learning for Mixup: Enhance Data Augmentation on Few-Shot Text Classification Tasks [75.42002070547267]
  We propose a self evolution learning (SE) based mixup approach for data augmentation in text classification. We introduce a novel instance specific label smoothing approach, which linearly interpolates the model's output and one hot labels of the original samples to generate new soft for label mixing up.
  arXiv  Detail & Related papers  (2023-05-22T23:43:23Z)
• Boosting Differentiable Causal Discovery via Adaptive Sample Reweighting [62.23057729112182]
  Differentiable score-based causal discovery methods learn a directed acyclic graph from observational data. We propose a model-agnostic framework to boost causal discovery performance by dynamically learning the adaptive weights for the Reweighted Score function, ReScore.
  arXiv  Detail & Related papers  (2023-03-06T14:49:59Z)
• Simplicity Bias Leads to Amplified Performance Disparities [8.60453031364566]
  We show that SGD-trained models have a bias towards simplicity, leading them to prioritize learning a majority class. A model may prioritize any class or group of the dataset that it finds simple-at the expense of what it finds complex.
  arXiv  Detail & Related papers  (2022-12-13T15:24:41Z)
• Split-PU: Hardness-aware Training Strategy for Positive-Unlabeled Learning [42.26185670834855]
  Positive-Unlabeled (PU) learning aims to learn a model with rare positive samples and abundant unlabeled samples. This paper focuses on improving the commonly-used nnPU with a novel training pipeline.
  arXiv  Detail & Related papers  (2022-11-30T05:48:31Z)
• An Investigation of Why Overparameterization Exacerbates Spurious Correlations [98.3066727301239]
  We identify two key properties of the training data that drive this behavior. We show how the inductive bias of models towards "memorizing" fewer examples can cause over parameterization to hurt.
  arXiv  Detail & Related papers  (2020-05-09T01:59:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.