Focusing Image Generation to Mitigate Spurious Correlations

• URL: http://arxiv.org/abs/2412.19457v1
• Date: Fri, 27 Dec 2024 04:48:56 GMT
• Title: Focusing Image Generation to Mitigate Spurious Correlations
• Authors: Xuewei Li, Zhenzhen Nie, Mei Yu, Zijian Zhang, Jie Gao, Tianyi Xu, Zhiqiang Liu,
• Abstract summary: Instance features in images exhibit spurious correlations with background features, affecting the training process of deep neural classifiers. We propose a data augmentation method called Spurious Correlations Guided Synthesis (SCGS) that mitigates spurious correlations through image generation model.
• Score: 13.225738145621843
• License:
• Abstract: Instance features in images exhibit spurious correlations with background features, affecting the training process of deep neural classifiers. This leads to insufficient attention to instance features by the classifier, resulting in erroneous classification outcomes. In this paper, we propose a data augmentation method called Spurious Correlations Guided Synthesis (SCGS) that mitigates spurious correlations through image generation model. This approach does not require expensive spurious attribute (group) labels for the training data and can be widely applied to other debiasing methods. Specifically, SCGS first identifies the incorrect attention regions of a pre-trained classifier on the training images, and then uses an image generation model to generate new training data based on these incorrect attended regions. SCGS increases the diversity and scale of the dataset to reduce the impact of spurious correlations on classifiers. Changes in the classifier's attention regions and experimental results on three different domain datasets demonstrate that this method is effective in reducing the classifier's reliance on spurious correlations.

Related papers

• Spuriousness-Aware Meta-Learning for Learning Robust Classifiers [26.544938760265136]
  Spurious correlations are brittle associations between certain attributes of inputs and target variables. Deep image classifiers often leverage them for predictions, leading to poor generalization on the data where the correlations do not hold. Mitigating the impact of spurious correlations is crucial towards robust model generalization, but it often requires annotations of the spurious correlations in data.
  arXiv  Detail & Related papers  (2024-06-15T21:41:25Z)
• Additional Look into GAN-based Augmentation for Deep Learning COVID-19 Image Classification [57.1795052451257]
  We study the dependence of the GAN-based augmentation performance on dataset size with a focus on small samples. We train StyleGAN2-ADA with both sets and then, after validating the quality of generated images, we use trained GANs as one of the augmentations approaches in multi-class classification problems. The GAN-based augmentation approach is found to be comparable with classical augmentation in the case of medium and large datasets but underperforms in the case of smaller datasets.
  arXiv  Detail & Related papers  (2024-01-26T08:28:13Z)
• Fine-grained Recognition with Learnable Semantic Data Augmentation [68.48892326854494]
  Fine-grained image recognition is a longstanding computer vision challenge. We propose diversifying the training data at the feature-level to alleviate the discriminative region loss problem. Our method significantly improves the generalization performance on several popular classification networks.
  arXiv  Detail & Related papers  (2023-09-01T11:15:50Z)
• Debiasing Counterfactuals In the Presence of Spurious Correlations [0.98342301244574]
  We introduce the first end-to-end training framework that integrates both (i) popular debiasing classifiers and (ii) counterfactual image generation. We demonstrate that the debiasing method: learns generalizable markers across the population, and (ii) successfully ignores spurious correlations and focuses on the underlying disease pathology.
  arXiv  Detail & Related papers  (2023-08-21T19:01:45Z)
• Controlling Learned Effects to Reduce Spurious Correlations in Text Classifiers [6.662800021628275]
  We propose an algorithm to regularize the learnt effect of the features on the model's prediction to the estimated effect of feature on label. On toxicity and IMDB review datasets, the proposed algorithm minimises spurious correlations and improves the minority group.
  arXiv  Detail & Related papers  (2023-05-26T12:15:54Z)
• Identifying Spurious Correlations and Correcting them with an Explanation-based Learning [4.039245878626345]
  We present a simple method to identify spurious correlations that have been learned by a model trained for image classification problems. We apply image-level perturbations and monitor changes in certainties of predictions made using the trained model.
  arXiv  Detail & Related papers  (2022-11-15T16:34:53Z)
• Treatment Learning Causal Transformer for Noisy Image Classification [62.639851972495094]
  In this work, we incorporate this binary information of "existence of noise" as treatment into image classification tasks to improve prediction accuracy. Motivated from causal variational inference, we propose a transformer-based architecture, that uses a latent generative model to estimate robust feature representations for noise image classification. We also create new noisy image datasets incorporating a wide range of noise factors for performance benchmarking.
  arXiv  Detail & Related papers  (2022-03-29T13:07:53Z)
• Learning from Partially Overlapping Labels: Image Segmentation under Annotation Shift [68.6874404805223]
  We propose several strategies for learning from partially overlapping labels in the context of abdominal organ segmentation. We find that combining a semi-supervised approach with an adaptive cross entropy loss can successfully exploit heterogeneously annotated data.
  arXiv  Detail & Related papers  (2021-07-13T09:22:24Z)
• Towards Robust Classification Model by Counterfactual and Invariant Data Generation [7.488317734152585]
  Spuriousness occurs when some features correlate with labels but are not causal. We propose two data generation processes to reduce spuriousness. Our data generations outperform state-of-the-art methods in accuracy when spurious correlations break.
  arXiv  Detail & Related papers  (2021-06-02T12:48:29Z)
• Background Splitting: Finding Rare Classes in a Sea of Background [55.03789745276442]
  We focus on the real-world problem of training accurate deep models for image classification of a small number of rare categories. In these scenarios, almost all images belong to the background category in the dataset (>95% of the dataset is background) We demonstrate that both standard fine-tuning approaches and state-of-the-art approaches for training on imbalanced datasets do not produce accurate deep models in the presence of this extreme imbalance.
  arXiv  Detail & Related papers  (2020-08-28T23:05:15Z)
• Out-of-distribution Generalization via Partial Feature Decorrelation [72.96261704851683]
  We present a novel Partial Feature Decorrelation Learning (PFDL) algorithm, which jointly optimize a feature decomposition network and the target image classification model. The experiments on real-world datasets demonstrate that our method can improve the backbone model's accuracy on OOD image classification datasets.
  arXiv  Detail & Related papers  (2020-07-30T05:48:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.