A Reverse Causal Framework to Mitigate Spurious Correlations for Debiasing Scene Graph Generation

• URL: http://arxiv.org/abs/2505.23451v1
• Date: Thu, 29 May 2025 13:57:01 GMT
• Title: A Reverse Causal Framework to Mitigate Spurious Correlations for Debiasing Scene Graph Generation
• Authors: Shuzhou Sun, Li Liu, Tianpeng Liu, Shuaifeng Zhi, Ming-Ming Cheng, Janne Heikkilä, Yongxiang Liu,
• Abstract summary: Scene Graph Generation (SGG) frameworks typically incorporate a detector to extract relationship features and a classifier to categorize these relationships.<n>Such a causal chain structure can yield spurious correlations between the detector's inputs and the final predictions.<n>We propose reconstructing the causal chain structure into a reverse causal structure, wherein the classifier's inputs are treated as the confounder.
• Score: 59.473751744275496
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing two-stage Scene Graph Generation (SGG) frameworks typically incorporate a detector to extract relationship features and a classifier to categorize these relationships; therefore, the training paradigm follows a causal chain structure, where the detector's inputs determine the classifier's inputs, which in turn influence the final predictions. However, such a causal chain structure can yield spurious correlations between the detector's inputs and the final predictions, i.e., the prediction of a certain relationship may be influenced by other relationships. This influence can induce at least two observable biases: tail relationships are predicted as head ones, and foreground relationships are predicted as background ones; notably, the latter bias is seldom discussed in the literature. To address this issue, we propose reconstructing the causal chain structure into a reverse causal structure, wherein the classifier's inputs are treated as the confounder, and both the detector's inputs and the final predictions are viewed as causal variables. Specifically, we term the reconstructed causal paradigm as the Reverse causal Framework for SGG (RcSGG). RcSGG initially employs the proposed Active Reverse Estimation (ARE) to intervene on the confounder to estimate the reverse causality, \ie the causality from final predictions to the classifier's inputs. Then, the Maximum Information Sampling (MIS) is suggested to enhance the reverse causality estimation further by considering the relationship information. Theoretically, RcSGG can mitigate the spurious correlations inherent in the SGG framework, subsequently eliminating the induced biases. Comprehensive experiments on popular benchmarks and diverse SGG frameworks show the state-of-the-art mean recall rate.

Related papers

• CAIFormer: A Causal Informed Transformer for Multivariate Time Series Forecasting [13.597882980144735]
  We propose an all-to-one forecasting paradigm that predicts each target variable separately.<n>The prediction relies solely on the first three causally relevant sub-segments.<n>Experiments on multiple benchmark datasets demonstrate the effectiveness of the CAIFormer.
  arXiv  Detail & Related papers  (2025-05-22T07:04:21Z)
• A Causal Adjustment Module for Debiasing Scene Graph Generation [28.44150555570101]
  We employ causal inference techniques to model the causality among skewed distributions.<n>Our method enables the composition of zero-shot relationships, thereby enhancing the model's ability to recognize such relationships.
  arXiv  Detail & Related papers  (2025-03-22T20:44:01Z)
• Inducing Causal Structure for Abstractive Text Summarization [76.1000380429553]
  We introduce a Structural Causal Model (SCM) to induce the underlying causal structure of the summarization data. We propose a Causality Inspired Sequence-to-Sequence model (CI-Seq2Seq) to learn the causal representations that can mimic the causal factors. Experimental results on two widely used text summarization datasets demonstrate the advantages of our approach.
  arXiv  Detail & Related papers  (2023-08-24T16:06:36Z)
• Unbiased Scene Graph Generation via Two-stage Causal Modeling [11.390135312161044]
  We propose Two-stage Causal Modeling (TsCM) for the Scene Graph Generation (SGG) task. TsCM takes the long-tailed distribution and semantic confusion as confounders to the Structural Causal Model (SCM) and then decouples the causal intervention into two stages. Our method can achieve state-of-the-art performance in terms of mean recall rate.
  arXiv  Detail & Related papers  (2023-07-11T14:11:24Z)
• Causal Autoregressive Flows [4.731404257629232]
  We highlight an intrinsic correspondence between a simple family of autoregressive normalizing flows and identifiable causal models. We exploit the fact that autoregressive flow architectures define an ordering over variables, analogous to a causal ordering, to show that they are well-suited to performing a range of causal inference tasks.
  arXiv  Detail & Related papers  (2020-11-04T13:17:35Z)
• Latent Causal Invariant Model [128.7508609492542]
  Current supervised learning can learn spurious correlation during the data-fitting process. We propose a Latent Causal Invariance Model (LaCIM) which pursues causal prediction.
  arXiv  Detail & Related papers  (2020-11-04T10:00:27Z)
• CASTLE: Regularization via Auxiliary Causal Graph Discovery [89.74800176981842]
  We introduce Causal Structure Learning (CASTLE) regularization and propose to regularize a neural network by jointly learning the causal relationships between variables. CASTLE efficiently reconstructs only the features in the causal DAG that have a causal neighbor, whereas reconstruction-based regularizers suboptimally reconstruct all input features.
  arXiv  Detail & Related papers  (2020-09-28T09:49:38Z)
• PCPL: Predicate-Correlation Perception Learning for Unbiased Scene Graph Generation [58.98802062945709]
  We propose a novel Predicate-Correlation Perception Learning scheme to adaptively seek out appropriate loss weights. Our PCPL framework is further equipped with a graph encoder module to better extract context features.
  arXiv  Detail & Related papers  (2020-09-02T08:30:09Z)
• CausalVAE: Structured Causal Disentanglement in Variational Autoencoder [52.139696854386976]
  The framework of variational autoencoder (VAE) is commonly used to disentangle independent factors from observations. We propose a new VAE based framework named CausalVAE, which includes a Causal Layer to transform independent factors into causal endogenous ones. Results show that the causal representations learned by CausalVAE are semantically interpretable, and their causal relationship as a Directed Acyclic Graph (DAG) is identified with good accuracy.
  arXiv  Detail & Related papers  (2020-04-18T20:09:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.