Causal Scaffolding for Physical Reasoning: A Benchmark for Causally-Informed Physical World Understanding in VLMs

• URL: http://arxiv.org/abs/2606.05966v1
• Date: Thu, 04 Jun 2026 10:07:05 GMT
• Title: Causal Scaffolding for Physical Reasoning: A Benchmark for Causally-Informed Physical World Understanding in VLMs
• Abstract summary: CausalPhys is a benchmark of over 3,000 carefully curated video- and image-based questions spanning four domains: Perception, Anticipation, Intervention, and Goal Orientation.<n>Each question is paired with an expert-annotated causal graph capturing object-attribute-event dependencies, enabling interpretable and fine-grained evaluation of causal understanding.<n>Building on this, we formulate a causal-graph-grounded metric that quantitatively measures how well a model's chain-of-thought reasoning aligns with the correct causal relations.<n>We propose Causal Rationale-informed Fine-Tuning (CRFT), which explicitly
• Score: 49.55219052565761
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Understanding and reasoning about the physical world is the foundation of intelligent behavior, yet state-of-the-art vision-language models (VLMs) still fail at causal physical reasoning, often producing plausible but incorrect answers. To address this gap, we introduce CausalPhys, a benchmark of over 3,000 carefully curated video- and image-based questions spanning four domains: Perception, Anticipation, Intervention, and Goal Orientation. Each question is paired with an expert-annotated causal graph capturing object-attribute-event dependencies, enabling interpretable and fine-grained evaluation of causal understanding. Building on this, we formulate a causal-graph-grounded metric that quantitatively measures how well a model's chain-of-thought reasoning aligns with the correct causal relations, moving beyond answer-only accuracy and enabling systematic diagnosis of VLMs' causal reasoning failures. Using this metric, we conduct a comprehensive analysis of leading VLMs, revealing systematic gaps in capturing causal dependencies and underscoring the need for causality-aware learning. To address these limitations, we further propose Causal Rationale-informed Fine-Tuning (CRFT), which explicitly aligns VLM reasoning with causal structures. Extensive experiments demonstrate that CRFT substantially enhances both reasoning accuracy and interpretability across multiple model backbones. By unifying dataset curation, causal evaluation, and causality-informed learning, CausalPhys establishes a strong foundation for advancing modern VLMs toward causally grounded physical reasoning.

Related papers

• Causal-Audit: Explicit and Auditable Graph-based Reasoning via Target-Aware Causal Chain Construction [14.96995835279229]
  Causal and intervention-based question answering is fundamental to advancing large language models.<n>We propose an explicit and auditable causal reasoning framework for context-free intervention-based question answering.
  arXiv  Detail & Related papers  (2026-04-22T23:24:18Z)
• Structured Causal Video Reasoning via Multi-Objective Alignment [102.61829546891543]
  We propose a compact representation of salient events and their causal relationships, which we name Structured Event Facts, prior to the reasoning stage.<n>This structured prior serves as an explicit constraint to promote concise and causally grounded reasoning.<n>We introduce CausalFact-60K and a four-stage training pipeline comprising facts alignment, format warm-start, thinking warm-start, and reinforcement learning-based post-training.
  arXiv  Detail & Related papers  (2026-04-06T04:49:30Z)
• Diagnosing Causal Reasoning in Vision-Language Models via Structured Relevance Graphs [18.83755844366017]
  We introduce Vision-Language Causal Graphs (VLCGs), a structured, query-conditioned representation that explicitly encodes causally relevant objects, attributes, relations, and scene-grounded assumptions.<n>We present ViLCaR, a diagnostic benchmark comprising tasks for Causal Attribution, Causal Inference, and Question Answering, along with graph-aligned evaluation metrics.<n> Experiments in state-of-the-art LVLMs show that injecting structured relevance information significantly improves attribution and inference compared to zero-shot and standard in-context learning.
  arXiv  Detail & Related papers  (2026-02-24T13:20:07Z)
• CausalFlip: A Benchmark for LLM Causal Judgment Beyond Semantic Matching [50.65932158912512]
  We propose a new causal reasoning benchmark, CausalFlip, to encourage the development of new large language models.<n>CaulFlip consists of causal judgment questions built over event triples that could form different confounder, chain, and collider relations.<n>We evaluate LLMs under multiple training paradigms, including answer-only training, explicit Chain-of-Thought supervision, and a proposed internalized causal reasoning approach.
  arXiv  Detail & Related papers  (2026-02-23T18:06:15Z)
• Do Large Language Models Show Biases in Causal Learning? Insights from Contingency Judgment [0.1547863211792184]
  Causal learning is the cognitive process of developing the capability of making causal inferences.<n>This process is prone to errors and biases, such as the illusion of causality.<n>This cognitive bias has been proposed to underlie many societal problems.
  arXiv  Detail & Related papers  (2025-10-15T18:09:00Z)
• COLD: Causal reasOning in cLosed Daily activities [7.782872276680731]
  We propose the COLD (Causal reasOning in cLosed Daily activities) framework.<n>It is built upon human understanding of daily real-world activities to reason about the causal nature of events.<n>We show that the proposed framework facilitates the creation of enormous causal queries.
  arXiv  Detail & Related papers  (2024-11-29T06:37:13Z)
• Failure Modes of LLMs for Causal Reasoning on Narratives [51.19592551510628]
  We investigate the interaction between world knowledge and logical reasoning.<n>We find that state-of-the-art large language models (LLMs) often rely on superficial generalizations.<n>We show that simple reformulations of the task can elicit more robust reasoning behavior.
  arXiv  Detail & Related papers  (2024-10-31T12:48:58Z)
• Context-Aware Reasoning On Parametric Knowledge for Inferring Causal Variables [49.31233968546582]
  We introduce a novel benchmark where the objective is to complete a partial causal graph.<n>We show the strong ability of LLMs to hypothesize the backdoor variables between a cause and its effect.<n>Unlike simple memorization of fixed associations, our task requires the LLM to reason according to the context of the entire graph.
  arXiv  Detail & Related papers  (2024-09-04T10:37:44Z)
• Cause and Effect: Can Large Language Models Truly Understand Causality? [1.2334534968968969]
  This research proposes a novel architecture called Context Aware Reasoning Enhancement with Counterfactual Analysis(CARE CA) framework. The proposed framework incorporates an explicit causal detection module with ConceptNet and counterfactual statements, as well as implicit causal detection through Large Language Models. The knowledge from ConceptNet enhances the performance of multiple causal reasoning tasks such as causal discovery, causal identification and counterfactual reasoning.
  arXiv  Detail & Related papers  (2024-02-28T08:02:14Z)
• Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learning [76.00395335702572]
  A central goal for AI and causality is the joint discovery of abstract representations and causal structure. Existing environments for studying causal induction are poorly suited for this objective because they have complicated task-specific causal graphs. In this work, our goal is to facilitate research in learning representations of high-level variables as well as causal structures among them.
  arXiv  Detail & Related papers  (2021-07-02T05:44:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.