The Abstraction Gap in Vision-Language Causal Reasoning

• URL: http://arxiv.org/abs/2605.28779v1
• Date: Wed, 27 May 2026 17:38:10 GMT
• Title: The Abstraction Gap in Vision-Language Causal Reasoning
• Abstract summary: Vision-language models (VLMs) generate fluent causal explanations.<n>Current evaluations cannot distinguish linguistic plausibility from faithful causal reasoning.<n>We introduce a dual-probe methodology that isolates these properties.
• Score: 0.5156484100374058
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-language models (VLMs) generate fluent causal explanations, but current evaluations cannot distinguish linguistic plausibility from faithful causal reasoning. We introduce a dual-probe methodology that isolates these properties. The Text-Only Probe measures linguistic quality. The Chain-Text Probe requires models to first generate explicit causal chains. The Abstraction Gap (AG) metric quantifies the normalized performance difference. Evaluating eight VLMs on CAGE (Causal Abstraction Gap Evaluation), a benchmark of 49,500 questions across 5,500 images spanning Pearl's causal hierarchy, we find seven models exhibit AG exceeding 0.50 with text scores of 6--8 but chain scores below 2.5. Fine-tuning on 45,000 chain-annotated examples fails to close the gap. However, one model achieves near-zero AG. The capability exists within current VLM architectures and depends on pretraining and architectural choices. CAGE provides a diagnostic tool for assessing faithful causal reasoning in VLMs.

Related papers

• NovGauge: A Fine-Grained Benchmark for Diagnosing LLMs' Capability in Paper Novelty Assessment [54.4265627247104]
  Large language models (LLMs) are increasingly used in peer review at major AI conferences.<n>Existing benchmarks assess novelty as a single holistic score.<n>We present NovGauge, a human-anchored benchmark for fine-grained novelty assessment diagnosis.
  arXiv  Detail & Related papers  (2026-09-10T08:34:56Z)
• A Hybrid Vision-Language Architecture for Automated Defect Reasoning and Report Generation in Industrial Inspection [0.42970700836450487]
  This paper describes a decoupled, edge-deployable pipeline for wind turbine blade inspection.<n>The Eyes a Y26-x-obb oriented bounding-box localizes defects at dataset-native resolution.<n>The Bridge a deterministic, parameter-free encoding module maps each detected bounding box to grid-referenced tokens.
  arXiv  Detail & Related papers  (2026-05-26T04:27:38Z)
• Seirênes: Adversarial Self-Play with Evolving Distractions for LLM Reasoning [56.48520300004217]
  We present Seirnes, a self-play RL framework that transforms contextual interference into an internal training signal.<n>A single model is trained to both construct plausible yet distracting contexts that expose its own reasoning blind spots.<n>By pitting these competing objectives against each other, Seirnes compels the model to move beyond superficial pattern matching.
  arXiv  Detail & Related papers  (2026-05-12T06:58:35Z)
• METER: Evaluating Multi-Level Contextual Causal Reasoning in Large Language Models [61.33372454250959]
  Contextual causal reasoning is a critical yet challenging capability for Large Language Models.<n>Existing benchmarks fail to ensure context consistency or cover the full causal hierarchy.<n>We pioneer METER to systematically benchmark LLMs across all three levels of the causal ladder.
  arXiv  Detail & Related papers  (2026-04-13T14:07:11Z)
• GroundCount: Grounding Vision-Language Models with Object Detection for Mitigating Counting Hallucinations [19.262982037123447]
  Vision Language Models (VLMs) exhibit persistent hallucinations in counting tasks, with accuracy substantially lower than other visual reasoning tasks.<n>We propose GroundCount, a framework that augments VLMs with explicit spatial grounding from object detection models to mitigate hallucinations.
  arXiv  Detail & Related papers  (2026-03-11T17:04:30Z)
• TACIT Benchmark: A Programmatic Visual Reasoning Benchmark for Generative and Discriminative Models [0.0]
  TACIT Benchmark is a programmatic visual reasoning benchmark comprising 10 tasks across 6 reasoning domains.<n>The benchmark provides dual-track evaluation: a generative track in which models must produce solution images verified through deterministic computer-vision pipelines, and a discriminative track offering five-way multiple choice with structurally plausible near-miss distractors.
  arXiv  Detail & Related papers  (2026-02-27T11:45:26Z)
• MMErroR: A Benchmark for Erroneous Reasoning in Vision-Language Models [29.830224745428566]
  We present MMErroR, a benchmark of 2,013 samples each embedding a single coherent reasoning error.<n>Unlike existing benchmarks that focus on answer correctness, MMErroR targets a process-level, error-centric evaluation.<n>We evaluate 20 advanced Vision-Language Models, even the best model (Gemini-3.0-Pro) classifies the error in only 66.47% of cases.
  arXiv  Detail & Related papers  (2026-01-06T17:45:26Z)
• Token-Level Inference-Time Alignment for Vision-Language Models [58.41370989069588]
  Vision-Language Models (VLMs) have become essential backbones of modern multimodal intelligence.<n>We present TITA, a lightweight framework that freezes the base VLM and instead trains a reward model to approximate its distribution.<n>During inference, implicit preference signals are extracted as log-probability ratios between the reward model and the target VLM, yielding dense autoregressive feedback.
  arXiv  Detail & Related papers  (2025-10-20T09:58:03Z)
• VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Models [121.03333569013148]
  We introduce VisuLogic: a benchmark of 1,000 human-verified problems across six categories.<n>These types of questions can be evaluated to assess the visual reasoning capabilities of MLLMs from multiple perspectives.<n>Most models score below 30% accuracy-only slightly above the 25% random baseline and far below the 51.4% achieved by humans.
  arXiv  Detail & Related papers  (2025-04-21T17:59:53Z)
• Beyond Chains of Thought: Benchmarking Latent-Space Reasoning Abilities in Large Language Models [0.0]
  Large language models (LLMs) can perform reasoning computations both internally within their latent space and externally.<n>This study introduces a benchmark designed to quantify model-internal reasoning in different domains.
  arXiv  Detail & Related papers  (2025-04-14T18:15:27Z)
• ExpliCa: Evaluating Explicit Causal Reasoning in Large Language Models [75.05436691700572]
  We introduce ExpliCa, a new dataset for evaluating Large Language Models (LLMs) in explicit causal reasoning.<n>We tested seven commercial and open-source LLMs on ExpliCa through prompting and perplexity-based metrics.<n>Surprisingly, models tend to confound temporal relations with causal ones, and their performance is also strongly influenced by the linguistic order of the events.
  arXiv  Detail & Related papers  (2025-02-21T14:23:14Z)
• Multimodal Chain-of-Thought Reasoning in Language Models [94.70184390935661]
  We propose Multimodal-CoT that incorporates language (text) and vision (images) modalities into a two-stage framework. Experimental results on ScienceQA and A-OKVQA benchmark datasets show the effectiveness of our proposed approach.
  arXiv  Detail & Related papers  (2023-02-02T07:51:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.