Identifying Spurious Correlations using Counterfactual Alignment

• URL: http://arxiv.org/abs/2312.02186v2
• Date: Tue, 01 Oct 2024 04:39:14 GMT
• Title: Identifying Spurious Correlations using Counterfactual Alignment
• Authors: Joseph Paul Cohen, Louis Blankemeier, Akshay Chaudhari,
• Abstract summary: Models driven by spurious correlations often yield poor generalization performance. We propose the counterfactual (CF) alignment method to detect and quantify spurious correlations.
• Score: 5.782952470371709
• License:
• Abstract: Models driven by spurious correlations often yield poor generalization performance. We propose the counterfactual (CF) alignment method to detect and quantify spurious correlations of black box classifiers. Our methodology is based on counterfactual images generated with respect to one classifier being input into other classifiers to see if they also induce changes in the outputs of these classifiers. The relationship between these responses can be quantified and used to identify specific instances where a spurious correlation exists. This is validated by observing intuitive trends in a face-attribute face-attribute and waterbird classifiers, as well as by fabricating spurious correlations and detecting their presence, both visually and quantitatively. Furthermore, utilizing the CF alignment method, we demonstrate that we can evaluate robust optimization methods (GroupDRO, JTT, and FLAC) by detecting a reduction in spurious correlations.

Related papers

• Spuriousness-Aware Meta-Learning for Learning Robust Classifiers [26.544938760265136]
  Spurious correlations are brittle associations between certain attributes of inputs and target variables. Deep image classifiers often leverage them for predictions, leading to poor generalization on the data where the correlations do not hold. Mitigating the impact of spurious correlations is crucial towards robust model generalization, but it often requires annotations of the spurious correlations in data.
  arXiv  Detail & Related papers  (2024-06-15T21:41:25Z)
• Learning Robust Classifiers with Self-Guided Spurious Correlation Mitigation [26.544938760265136]
  Deep neural classifiers rely on spurious correlations between spurious attributes of inputs and targets to make predictions. We propose a self-guided spurious correlation mitigation framework. We show that training the classifier to distinguish different prediction behaviors reduces its reliance on spurious correlations without knowing them a priori.
  arXiv  Detail & Related papers  (2024-05-06T17:12:21Z)
• Detecting Spurious Correlations via Robust Visual Concepts in Real and AI-Generated Image Classification [12.992095539058022]
  We introduce a general-purpose method that efficiently detects potential spurious correlations. The proposed method provides intuitive explanations while eliminating the need for pixel-level annotations. Our method is also suitable for detecting spurious correlations that may propagate to downstream applications originating from generative models.
  arXiv  Detail & Related papers  (2023-11-03T01:12:35Z)
• Understanding and Mitigating Spurious Correlations in Text Classification with Neighborhood Analysis [69.07674653828565]
  Machine learning models have a tendency to leverage spurious correlations that exist in the training set but may not hold true in general circumstances. In this paper, we examine the implications of spurious correlations through a novel perspective called neighborhood analysis. We propose a family of regularization methods, NFL (doN't Forget your Language) to mitigate spurious correlations in text classification.
  arXiv  Detail & Related papers  (2023-05-23T03:55:50Z)
• Learning Partial Correlation based Deep Visual Representation for Image Classification [61.0532370259644]
  We formulate sparse inverse covariance estimation (SICE) as a novel structured layer of CNN. Our work obtains a partial correlation based deep visual representation and mitigates the small sample problem. Experiments show the efficacy and superior classification performance of our model.
  arXiv  Detail & Related papers  (2023-04-23T10:09:01Z)
• FECANet: Boosting Few-Shot Semantic Segmentation with Feature-Enhanced Context-Aware Network [48.912196729711624]
  Few-shot semantic segmentation is the task of learning to locate each pixel of a novel class in a query image with only a few annotated support images. We propose a Feature-Enhanced Context-Aware Network (FECANet) to suppress the matching noise caused by inter-class local similarity. In addition, we propose a novel correlation reconstruction module that encodes extra correspondence relations between foreground and background and multi-scale context semantic features.
  arXiv  Detail & Related papers  (2023-01-19T16:31:13Z)
• Causal Transportability for Visual Recognition [70.13627281087325]
  We show that standard classifiers fail because the association between images and labels is not transportable across settings. We then show that the causal effect, which severs all sources of confounding, remains invariant across domains. This motivates us to develop an algorithm to estimate the causal effect for image classification.
  arXiv  Detail & Related papers  (2022-04-26T15:02:11Z)
• Visualizing Classifier Adjacency Relations: A Case Study in Speaker Verification and Voice Anti-Spoofing [72.4445825335561]
  We propose a simple method to derive 2D representation from detection scores produced by an arbitrary set of binary classifiers. Based upon rank correlations, our method facilitates a visual comparison of classifiers with arbitrary scores. While the approach is fully versatile and can be applied to any detection task, we demonstrate the method using scores produced by automatic speaker verification and voice anti-spoofing systems.
  arXiv  Detail & Related papers  (2021-06-11T13:03:33Z)
• Identifying Spurious Correlations for Robust Text Classification [9.457737910527829]
  We propose a method to distinguish spurious and genuine correlations in text classification. We use features derived from treatment effect estimators to distinguish spurious correlations from "genuine" ones. Experiments on four datasets suggest that using this approach to inform feature selection also leads to more robust classification.
  arXiv  Detail & Related papers  (2020-10-06T03:49:22Z)
• Out-of-distribution Generalization via Partial Feature Decorrelation [72.96261704851683]
  We present a novel Partial Feature Decorrelation Learning (PFDL) algorithm, which jointly optimize a feature decomposition network and the target image classification model. The experiments on real-world datasets demonstrate that our method can improve the backbone model's accuracy on OOD image classification datasets.
  arXiv  Detail & Related papers  (2020-07-30T05:48:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.