DrawingVQA: A Real-World Benchmark for Multi-Depth Visual-Textual Reasoning on Construction Drawings

• URL: http://arxiv.org/abs/2607.15418v1
• Date: Thu, 16 Jul 2026 19:44:03 GMT
• Title: DrawingVQA: A Real-World Benchmark for Multi-Depth Visual-Textual Reasoning on Construction Drawings
• Abstract summary: DrawingVQA is the first benchmark designed to evaluate multimodal large language models (MLLMs) on real-world construction drawings.<n>DrawingVQA bridges this gap with 33 "Issued for Construction" drawings and 92 expertly curated question-answer pairs.
• Score: 0.4078247440919472
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We introduce DrawingVQA, the first benchmark designed to evaluate multimodal large language models (MLLMs) on real-world construction drawings -- a core media in architecture, civil, and many other engineering practices. Unlike natural images or schematic floor plans, construction drawings fuse abstract geometry, symbolic notation, tabular data, annotations, and domain-specific text, forming a uniquely complex visual-textual domain core to engineering workflows. DrawingVQA bridges this gap with 33 "Issued for Construction" drawings and 92 expertly curated question-answer pairs, spanning three reasoning depths: perceptual understanding, contextual interpretation, and domain-expert reasoning. To evaluate model capabilities, we present a dual categorization framework to jointly analyze performance across seven construction-engineering and four MLLM capability dimensions -- the first to explicitly map engineering workflows to AI reasoning competencies. Evaluations of state-of-the-art MLLMs reveal a substantial gap between model and expert performance, particularly at higher reasoning depths. This benchmark lays a foundation for domain-specialized multimodal reasoning to allow for advancement on integration of AI-driven understanding and real-world engineering workflows.

Related papers

• Symbolic and Abstractive Reasoning with Complex Visual Queries [57.59111649292774]
  We explore a novel abstract data type termed complex visual query (CVQ)<n>CVQ probes symbolic and abstractive reasoning, which is a critical yet underexplored dimension of human-like neuro-symbolic reasoning for MLLMs.<n>We introduce a two-stage training framework that progressively equips MLLMs with robust visual reasoning capabilities.
  arXiv  Detail & Related papers  (2026-06-08T08:30:51Z)
• Dual-Pathway Geometry-Aware MLLM for Spatial Intelligence [63.11502936651325]
  We propose GAMSI, a dual-pathway Geometry-Aware MLLM for Spatial Intelligence.<n>It takes only RGB images as input while internalizing both forms of geometric prior within a unified autoregressive backbone.<n>GAMSI achieves state-of-the-art performance on seven spatial intelligence benchmarks.
  arXiv  Detail & Related papers  (2026-05-25T01:33:19Z)
• Benchmarking Requirement-to-Architecture Generation with Hybrid Evaluation [8.867061253116367]
  Large Language Models (LLMs) have demonstrated significant potential in automating software engineering tasks.<n> Generating software architecture designs from requirement documents is a crucial step in software development.<n>R2ABench is a novel benchmark comprising diverse real-world software projects paired with comprehensive Product Requirements Documents (PRDs) and expert-curated PlantUML reference diagrams.
  arXiv  Detail & Related papers  (2026-04-08T04:58:36Z)
• Enginuity: Building an Open Multi-Domain Dataset of Complex Engineering Diagrams [2.8809775760443657]
  We propose Enginuity - the first open, large-scale, multi-domain engineering diagram dataset with comprehensive structural annotations designed for automated diagram parsing.<n>By capturing hierarchical component relationships, connections, and semantic elements across diverse engineering domains, our proposed dataset would enable multimodal large language models to address critical downstream tasks including structured diagram parsing, cross-modal information retrieval, and AI-assisted engineering simulation.
  arXiv  Detail & Related papers  (2026-01-19T18:54:50Z)
• CircuitSense: A Hierarchical Circuit System Benchmark Bridging Visual Comprehension and Symbolic Reasoning in Engineering Design Process [29.38618453695266]
  Engineering design operates through hierarchical abstraction from system specifications to component implementations.<n>While Multi-modal Large Language Models (MLLMs) excel at natural image tasks, their ability to extract mathematical models from technical diagrams remains unexplored.<n>We present textbfCircuitSense, a benchmark evaluating circuit understanding across this hierarchy through 8,006+ problems spanning component-level schematics to system-level block diagrams.
  arXiv  Detail & Related papers  (2025-09-26T13:32:14Z)
• SoM-1K: A Thousand-Problem Benchmark Dataset for Strength of Materials [16.756001896133757]
  We introduce SoM-1K, the first large-scale multimodal benchmark dataset dedicated to evaluating foundation models on problems in the strength of materials.<n>This work establishes a rigorous benchmark for engineering AI and highlights a critical need for developing more robust multimodal reasoning capabilities.
  arXiv  Detail & Related papers  (2025-09-25T12:28:22Z)
• Spatial Understanding from Videos: Structured Prompts Meet Simulation Data [89.77871049500546]
  We present a unified framework for enhancing 3D spatial reasoning in pre-trained vision-language models without modifying their architecture.<n>This framework combines SpatialMind, a structured prompting strategy that decomposes complex scenes and questions into interpretable reasoning steps, with ScanForgeQA, a scalable question-answering dataset built from diverse 3D simulation scenes.
  arXiv  Detail & Related papers  (2025-06-04T07:36:33Z)
• OmniParser V2: Structured-Points-of-Thought for Unified Visual Text Parsing and Its Generality to Multimodal Large Language Models [58.45517851437422]
  Visually-situated text parsing (VsTP) has recently seen notable advancements, driven by the growing demand for automated document understanding.<n>Existing solutions often rely on task-specific architectures and objectives for individual tasks.<n>In this paper, we introduce Omni V2, a universal model that unifies VsTP typical tasks, including text spotting, key information extraction, table recognition, and layout analysis.
  arXiv  Detail & Related papers  (2025-02-22T09:32:01Z)
• A Cognitive Paradigm Approach to Probe the Perception-Reasoning Interface in VLMs [3.2228025627337864]
  This paper introduces a structured evaluation framework to dissect the perception-reasoning interface in Vision-Language Models (VLMs)<n>We propose three distinct evaluation paradigms, mirroring human problem-solving strategies.<n>Applying this framework, we demonstrate that CA, leveraging powerful language models for reasoning over rich, independently generated descriptions, achieves new state-of-the-art (SOTA) performance.
  arXiv  Detail & Related papers  (2025-01-23T12:42:42Z)
• GLDesigner: Leveraging Multi-Modal LLMs as Designer for Enhanced Aesthetic Text Glyph Layouts [53.568057283934714]
  We propose a Vision-Language Model (VLM)-based framework that generates content-aware text logo layouts.<n>We introduce two model techniques that reduce the computational cost for processing multiple glyph images simultaneously.<n>To support instruction tuning of our model, we construct two extensive text logo datasets that are five times larger than existing public datasets.
  arXiv  Detail & Related papers  (2024-11-18T10:04:10Z)
• Bridging the Gap: Exploring the Capabilities of Bridge-Architectures for Complex Visual Reasoning Tasks [4.093474663507322]
  Bridge-architectures project from the image space to the text space to solve tasks such as VQA, captioning, and image retrieval. We extend the traditional bridge architectures for the NLVR2 dataset, by adding object level features to faciliate fine-grained object reasoning. Our analysis shows that adding object level features to bridge architectures does not help, and that pre-training on multi-modal data is key for good performance on complex reasoning tasks such as NLVR2.
  arXiv  Detail & Related papers  (2023-07-31T03:57:31Z)
• Universal Information Extraction as Unified Semantic Matching [54.19974454019611]
  We decouple information extraction into two abilities, structuring and conceptualizing, which are shared by different tasks and schemas. Based on this paradigm, we propose to universally model various IE tasks with Unified Semantic Matching framework. In this way, USM can jointly encode schema and input text, uniformly extract substructures in parallel, and controllably decode target structures on demand.
  arXiv  Detail & Related papers  (2023-01-09T11:51:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.