LADBench: A Benchmark for Logical Fault Detection in Images

• URL: http://arxiv.org/abs/2606.17433v1
• Date: Tue, 16 Jun 2026 02:32:38 GMT
• Title: LADBench: A Benchmark for Logical Fault Detection in Images
• Abstract summary: Large Vision Language Models (VLMs) excel at visual question answering and semantic grounding, but their capacity for autonomous logical reasoning remains underexplored.<n>Existing anomaly benchmarks emphasize visual errors or direct prompting rather than the physical and social common sense needed for open-world deployment.<n>We introduce LAD-bench, a benchmark of more than 1,000 curated synthetic images with logical anomalies across four domains.
• Score: 1.6882040908691864
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Vision Language Models (VLMs) excel at visual question answering and semantic grounding, but their capacity for autonomous logical reasoning remains underexplored. Existing anomaly benchmarks emphasize visual errors or direct prompting rather than the physical and social common sense needed for open-world deployment. To address this, we introduce LAD-bench, a benchmark of more than 1,000 curated synthetic images with logical anomalies across four domains: Residential, Urban, Collaborative, and Nature. We further propose a Tiered Prompting Protocol based on progressive disclosure, which measures how much explicit assistance a model needs to localize and reason about a logical fault. Evaluating leading foundation models reveals substantial weaknesses: even the best achieves only 70.11% overall accuracy, showing that implicit logical fault detection remains unsolved. Crucially, models often fail to identify anomalies even after receiving explicit hints in deeper tiers. By surfacing these limitations in sequential multimodal reasoning, LAD-Bench offers a rigorous framework for advancing the safety, reliability, and cognitive alignment of autonomous visual systems. Dataset and Code: https://huggingface.co/datasets/SahasraK/LADBench

Related papers

• LLM Reasoning for Subjective Tasks: Failure Modes, Mitigation, and Dynamic Reasoning Routing [7.632217365130212]
  We study whether explicit reasoning generalizes to subjective, human-centric industry rubrics.<n>We show that verification accuracy swings by nearly 0.38 macro-F1 depending solely on the adopted reasoning persona.<n>This work offers both a scalable algorithmic patch and a long-term architectural blueprint for aligning reasoning models with real-world subjective constraints.
  arXiv  Detail & Related papers  (2026-08-09T20:02:44Z)
• LARAD: Layout-Aware Road Anomaly Detection via Spatial-Logic Reasoning [16.92190487823839]
  We propose layout-Aware Road Anomaly Detection (LARAD), which shifts the paradigm from appearance matching to spatial-logic reasoning.<n>First, we introduce the Spatial-Logic Violation Synthesis pipeline, which generates training samples that are texture-consistent yet spatially invalid.<n>Second, we augment a standard closed-set segmentation network with a lightweight, OoD-guided attention branch.
  arXiv  Detail & Related papers  (2026-07-14T15:09:00Z)
• PixelEyes: Decoupling Perception and Reasoning for Pinpoint Visual Evidence Seeking [63.574885296709716]
  This paper explores multi-turn visual reasoning and observes that MLLMs repeatedly fail to localize the target.<n>We propose PixelEyes, a visual reasoning agent that explicitly decouples reasoning from perception.
  arXiv  Detail & Related papers  (2026-06-30T19:51:54Z)
• ERQA-Plus: A Diagnostic Benchmark for Reasoning in Embodied AI [14.957780321740394]
  ERQA-Plus is a diagnostic benchmark for reasoning in embodied AI.<n>It contains 1,766 question-answer instances grounded in 711 robot-centric images.
  arXiv  Detail & Related papers  (2026-06-16T07:56:33Z)
• Anatomy of a Lie: A Multi-Stage Diagnostic Framework for Tracing Hallucinations in Vision-Language Models [62.932580559941414]
  Vision-Language Models (VLMs) frequently "hallucinate" - generate plausible yet factually incorrect statements.<n>We propose a new paradigm for diagnosing hallucinations, recasting them from static output errors into dynamic pathologies of a model's computational cognition.
  arXiv  Detail & Related papers  (2026-03-16T17:20:38Z)
• TACIT Benchmark: A Programmatic Visual Reasoning Benchmark for Generative and Discriminative Models [0.0]
  TACIT Benchmark is a programmatic visual reasoning benchmark comprising 10 tasks across 6 reasoning domains.<n>The benchmark provides dual-track evaluation: a generative track in which models must produce solution images verified through deterministic computer-vision pipelines, and a discriminative track offering five-way multiple choice with structurally plausible near-miss distractors.
  arXiv  Detail & Related papers  (2026-02-27T11:45:26Z)
• LogicGraph : Benchmarking Multi-Path Logical Reasoning via Neuro-Symbolic Generation and Verification [24.91906506651266]
  We introduce LogicGraph, the first benchmark aimed to systematically evaluate multi-path logical reasoning.<n>This pipeline yields solver-verified reasoning problems formalized by high-depth multi-path reasoning.<n>We propose a reference-free evaluation framework to rigorously assess model performance in both convergent and divergent regimes.
  arXiv  Detail & Related papers  (2026-02-24T16:04:26Z)
• SpatiaLQA: A Benchmark for Evaluating Spatial Logical Reasoning in Vision-Language Models [60.088066516175026]
  We introduce a benchmark designed to evaluate the spatial logical reasoning capabilities of Vision-Language Models (VLMs)<n>We conduct extensive experiments on 41 mainstream VLMs, and the results show that even the most advanced models still struggle with spatial logical reasoning.<n>We propose a method called recursive scene graph assisted reasoning, which leverages visual foundation models to progressively decompose complex scenes into task-relevant scene graphs.
  arXiv  Detail & Related papers  (2026-02-24T13:38:37Z)
• Self-Refining Vision Language Model for Robotic Failure Detection and Reasoning [16.274791437311602]
  We introduce ARMOR: Adaptive Round-based Multi-task mOdel for Robotic failure detection and reasoning.<n>We formulate detection and reasoning as a multi-task self-refinement process.<n>We show that ARMOR achieves state-of-the-art performance by improving over the previous approaches by up to 30% on failure detection rate.
  arXiv  Detail & Related papers  (2026-02-12T20:55:36Z)
• Reason-IAD: Knowledge-Guided Dynamic Latent Reasoning for Explainable Industrial Anomaly Detection [85.29900916231655]
  Reason-IAD is a knowledge-guided dynamic latent reasoning framework for explainable industrial anomaly detection.<n>Experiments demonstrate that Reason-IAD consistently outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-02-10T14:54:17Z)
• Seeing Is Believing? A Benchmark for Multimodal Large Language Models on Visual Illusions and Anomalies [40.03295633717008]
  We introduce VIA-Bench, a benchmark designed to probe model performance on visual illusions and anomalies.<n>We construct over 1K high-quality question-answer pairs that require nuanced visual reasoning.<n>Our findings reveal a fundamental divergence between machine and human perception, suggesting that resolving such perceptual bottlenecks is critical for the advancement of artificial general intelligence.
  arXiv  Detail & Related papers  (2026-02-02T08:48:03Z)
• From Indoor to Open World: Revealing the Spatial Reasoning Gap in MLLMs [65.04549036809557]
  We introduce a benchmark built from pedestrian-perspective videos captured with synchronized stereo cameras, LiDAR, and IMU/GPS sensors.<n>This dataset provides metrically precise 3D information, enabling the automatic generation of spatial reasoning questions.<n> Evaluations reveal that the performance gains observed in structured indoor benchmarks vanish in open-world settings.
  arXiv  Detail & Related papers  (2025-12-22T18:58:12Z)
• VAR: Visual Attention Reasoning via Structured Search and Backtracking [49.427842994857635]
  We introduce Visual Attention Reasoning, a framework that recasts grounded reasoning as a structured search.<n> VAR decomposes the reasoning process into two key stages: traceable evidence grounding and search-based chain-of-thought.<n>We show that our 7B model, VAR-7B, sets a new state-of-the-art on a comprehensive suite of hallucination and safety benchmarks.
  arXiv  Detail & Related papers  (2025-10-21T13:18:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.