Detecting Spurious Correlations via Robust Visual Concepts in Real and
AI-Generated Image Classification
- URL: http://arxiv.org/abs/2311.01655v2
- Date: Thu, 16 Nov 2023 00:22:27 GMT
- Title: Detecting Spurious Correlations via Robust Visual Concepts in Real and
AI-Generated Image Classification
- Authors: Preetam Prabhu Srikar Dammu, Chirag Shah
- Abstract summary: We introduce a general-purpose method that efficiently detects potential spurious correlations.
The proposed method provides intuitive explanations while eliminating the need for pixel-level annotations.
Our method is also suitable for detecting spurious correlations that may propagate to downstream applications originating from generative models.
- Score: 12.992095539058022
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Often machine learning models tend to automatically learn associations
present in the training data without questioning their validity or
appropriateness. This undesirable property is the root cause of the
manifestation of spurious correlations, which render models unreliable and
prone to failure in the presence of distribution shifts. Research shows that
most methods attempting to remedy spurious correlations are only effective for
a model's known spurious associations. Current spurious correlation detection
algorithms either rely on extensive human annotations or are too restrictive in
their formulation. Moreover, they rely on strict definitions of visual
artifacts that may not apply to data produced by generative models, as they are
known to hallucinate contents that do not conform to standard specifications.
In this work, we introduce a general-purpose method that efficiently detects
potential spurious correlations, and requires significantly less human
interference in comparison to the prior art. Additionally, the proposed method
provides intuitive explanations while eliminating the need for pixel-level
annotations. We demonstrate the proposed method's tolerance to the peculiarity
of AI-generated images, which is a considerably challenging task, one where
most of the existing methods fall short. Consequently, our method is also
suitable for detecting spurious correlations that may propagate to downstream
applications originating from generative models.
Related papers
- Demystifying amortized causal discovery with transformers [21.058343547918053]
Supervised learning approaches for causal discovery from observational data often achieve competitive performance.
In this work, we investigate CSIvA, a transformer-based model promising to train on synthetic data and transfer to real data.
We bridge the gap with existing identifiability theory and show that constraints on the training data distribution implicitly define a prior on the test observations.
arXiv Detail & Related papers (2024-05-27T08:17:49Z) - Utilizing Adversarial Examples for Bias Mitigation and Accuracy Enhancement [3.0820287240219795]
We propose a novel approach to mitigate biases in computer vision models by utilizing counterfactual generation and fine-tuning.
Our approach leverages a curriculum learning framework combined with a fine-grained adversarial loss to fine-tune the model using adversarial examples.
We validate our approach through both qualitative and quantitative assessments, demonstrating improved bias mitigation and accuracy compared to existing methods.
arXiv Detail & Related papers (2024-04-18T00:41:32Z) - DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception [78.26734070960886]
Current perceptive models heavily depend on resource-intensive datasets.
We introduce perception-aware loss (P.A. loss) through segmentation, improving both quality and controllability.
Our method customizes data augmentation by extracting and utilizing perception-aware attribute (P.A. Attr) during generation.
arXiv Detail & Related papers (2024-03-20T04:58:03Z) - On the Universal Adversarial Perturbations for Efficient Data-free
Adversarial Detection [55.73320979733527]
We propose a data-agnostic adversarial detection framework, which induces different responses between normal and adversarial samples to UAPs.
Experimental results show that our method achieves competitive detection performance on various text classification tasks.
arXiv Detail & Related papers (2023-06-27T02:54:07Z) - ChiroDiff: Modelling chirographic data with Diffusion Models [132.5223191478268]
We introduce a powerful model-class namely "Denoising Diffusion Probabilistic Models" or DDPMs for chirographic data.
Our model named "ChiroDiff", being non-autoregressive, learns to capture holistic concepts and therefore remains resilient to higher temporal sampling rate.
arXiv Detail & Related papers (2023-04-07T15:17:48Z) - Less is More: Mitigate Spurious Correlations for Open-Domain Dialogue
Response Generation Models by Causal Discovery [52.95935278819512]
We conduct the first study on spurious correlations for open-domain response generation models based on a corpus CGDIALOG curated in our work.
Inspired by causal discovery algorithms, we propose a novel model-agnostic method for training and inference of response generation model.
arXiv Detail & Related papers (2023-03-02T06:33:48Z) - Causal Inference via Style Transfer for Out-of-distribution
Generalisation [10.998592702137858]
Out-of-distribution generalisation aims to build a model that can generalise well on an unseen target domain.
We propose a novel method that effectively deals with hidden confounders by successfully implementing front-door adjustment.
arXiv Detail & Related papers (2022-12-06T15:43:54Z) - Identification of Latent Variables From Graphical Model Residuals [0.0]
We present a novel method to control for the latent space when estimating a DAG by iteratively deriving proxies for the latent space from the residuals of the inferred model.
We show that any improvement of prediction of an outcome is intrinsically capped and cannot rise beyond a certain limit as compared to the confounded model.
arXiv Detail & Related papers (2021-01-07T02:28:49Z) - Proactive Pseudo-Intervention: Causally Informed Contrastive Learning
For Interpretable Vision Models [103.64435911083432]
We present a novel contrastive learning strategy called it Proactive Pseudo-Intervention (PPI)
PPI leverages proactive interventions to guard against image features with no causal relevance.
We also devise a novel causally informed salience mapping module to identify key image pixels to intervene, and show it greatly facilitates model interpretability.
arXiv Detail & Related papers (2020-12-06T20:30:26Z) - Learning Causal Models Online [103.87959747047158]
Predictive models can rely on spurious correlations in the data for making predictions.
One solution for achieving strong generalization is to incorporate causal structures in the models.
We propose an online algorithm that continually detects and removes spurious features.
arXiv Detail & Related papers (2020-06-12T20:49:20Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.