CAUSAL3D: A Comprehensive Benchmark for Causal Learning from Visual Data

• URL: http://arxiv.org/abs/2503.04852v1
• Date: Thu, 06 Mar 2025 03:40:01 GMT
• Title: CAUSAL3D: A Comprehensive Benchmark for Causal Learning from Visual Data
• Authors: Disheng Liu, Yiran Qiao, Wuche Liu, Yiren Lu, Yunlai Zhou, Tuo Liang, Yu Yin, Jing Ma,
• Abstract summary: We introduce textsctextbfCausal3D, a novel benchmark that integrates structured data (tables) with corresponding visual representations (images) to evaluate causal reasoning.<n>Causal3D comprises 19 3D-scene datasets capturing diverse causal relations, views, and backgrounds.
• Score: 10.435321637846142
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: True intelligence hinges on the ability to uncover and leverage hidden causal relations. Despite significant progress in AI and computer vision (CV), there remains a lack of benchmarks for assessing models' abilities to infer latent causality from complex visual data. In this paper, we introduce \textsc{\textbf{Causal3D}}, a novel and comprehensive benchmark that integrates structured data (tables) with corresponding visual representations (images) to evaluate causal reasoning. Designed within a systematic framework, Causal3D comprises 19 3D-scene datasets capturing diverse causal relations, views, and backgrounds, enabling evaluations across scenes of varying complexity. We assess multiple state-of-the-art methods, including classical causal discovery, causal representation learning, and large/vision-language models (LLMs/VLMs). Our experiments show that as causal structures grow more complex without prior knowledge, performance declines significantly, highlighting the challenges even advanced methods face in complex causal scenarios. Causal3D serves as a vital resource for advancing causal reasoning in CV and fostering trustworthy AI in critical domains.

Related papers

• Look, Learn and Leverage (L$^3$): Mitigating Visual-Domain Shift and Discovering Intrinsic Relations via Symbolic Alignment [19.700374722227107]
  We propose a novel learning framework, Look, Learn and Leverage (L$3$), which decomposes the learning process into three distinct phases. A relations discovery model can be trained on the source domain, and when the visual domain shifts and the intrinsic relations are absent, the pretrained relations discovery model can be directly reused and maintain a satisfactory performance.
  arXiv  Detail & Related papers  (2024-08-30T15:53:48Z)
• Enhancing Generalizability of Representation Learning for Data-Efficient 3D Scene Understanding [50.448520056844885]
  We propose a generative Bayesian network to produce diverse synthetic scenes with real-world patterns. A series of experiments robustly display our method's consistent superiority over existing state-of-the-art pre-training approaches.
  arXiv  Detail & Related papers  (2024-06-17T07:43:53Z)
• SSL Framework for Causal Inconsistency between Structures and Representations [31.895570222735955]
  Cross-pollination between causal discovery and deep learning has led to increasingly extensive interactions.<n>Indefinite Data has conflicts between causal relationships expressed by the causal structure and causal representation generated by deep learning models.<n>To alleviate causal inconsistency, we proposed a self-supervised learning framework based on intervention.
  arXiv  Detail & Related papers  (2023-10-28T08:29:49Z)
• Inducing Causal Structure for Abstractive Text Summarization [76.1000380429553]
  We introduce a Structural Causal Model (SCM) to induce the underlying causal structure of the summarization data. We propose a Causality Inspired Sequence-to-Sequence model (CI-Seq2Seq) to learn the causal representations that can mimic the causal factors. Experimental results on two widely used text summarization datasets demonstrate the advantages of our approach.
  arXiv  Detail & Related papers  (2023-08-24T16:06:36Z)
• Context De-confounded Emotion Recognition [12.037240778629346]
  Context-Aware Emotion Recognition (CAER) aims to perceive the emotional states of the target person with contextual information. A long-overlooked issue is that a context bias in existing datasets leads to a significantly unbalanced distribution of emotional states. This paper provides a causality-based perspective to disentangle the models from the impact of such bias, and formulate the causalities among variables in the CAER task.
  arXiv  Detail & Related papers  (2023-03-21T15:12:20Z)
• Causal Triplet: An Open Challenge for Intervention-centric Causal Representation Learning [98.78136504619539]
  Causal Triplet is a causal representation learning benchmark featuring visually more complex scenes. We show that models built with the knowledge of disentangled or object-centric representations significantly outperform their distributed counterparts.
  arXiv  Detail & Related papers  (2023-01-12T17:43:38Z)
• Cross-Modal Causal Relational Reasoning for Event-Level Visual Question Answering [134.91774666260338]
  Existing visual question answering methods often suffer from cross-modal spurious correlations and oversimplified event-level reasoning processes. We propose a framework for cross-modal causal relational reasoning to address the task of event-level visual question answering.
  arXiv  Detail & Related papers  (2022-07-26T04:25:54Z)
• Causal Reasoning Meets Visual Representation Learning: A Prospective Study [117.08431221482638]
  Lack of interpretability, robustness, and out-of-distribution generalization are becoming the challenges of the existing visual models. Inspired by the strong inference ability of human-level agents, recent years have witnessed great effort in developing causal reasoning paradigms. This paper aims to provide a comprehensive overview of this emerging field, attract attention, encourage discussions, bring to the forefront the urgency of developing novel causal reasoning methods.
  arXiv  Detail & Related papers  (2022-04-26T02:22:28Z)
• Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learning [76.00395335702572]
  A central goal for AI and causality is the joint discovery of abstract representations and causal structure. Existing environments for studying causal induction are poorly suited for this objective because they have complicated task-specific causal graphs. In this work, our goal is to facilitate research in learning representations of high-level variables as well as causal structures among them.
  arXiv  Detail & Related papers  (2021-07-02T05:44:56Z)
• Visual Causality Analysis of Event Sequence Data [32.74361488457415]
  We introduce a visual analytics method for recovering causalities in event sequence data. We extend the Granger causality analysis algorithm on Hawkes processes to incorporate user feedback into causal model refinement.
  arXiv  Detail & Related papers  (2020-09-01T04:28:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.