ArchSIBench: Benchmarking the Architectural Spatial Intelligence of Vision-Language Models

• URL: http://arxiv.org/abs/2605.20837v1
• Date: Wed, 20 May 2026 07:27:08 GMT
• Title: ArchSIBench: Benchmarking the Architectural Spatial Intelligence of Vision-Language Models
• Abstract summary: ArchSIBench is a Benchmark for Architectural Spatial Intelligence based on the perspectives from architecture, cognitive science, and psychology.<n> ArchSIBench covers five core dimensions: perception, reasoning, navigation, transformation, and configuration, comprising 17 fine-grained subtasks.<n>We evaluate various Vision-Language Models (VLMs) and find that the architectural spatial intelligence of most models shows significant differences from human baselines.
• Score: 16.656416066183887
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Architectural spatial intelligence, the ability to recognize and infer architectural space, is fundamental to tasks such as robot navigation, embodied interaction, and 3D scene understanding and generation. Although extensive research has evaluated the basic spatial skills of Vision-Language Models (VLMs) such as relative orientation, distance comparison, and object counting, these tasks cover only the most elementary levels of spatial cognition and largely overlook higher-level cognition of architectural space, including layout understanding, circulation patterns, and functional zoning. In this work, we present ArchSIBench, a Benchmark for Architectural Spatial Intelligence based on the perspectives from architecture, cognitive science, and psychology. ArchSIBench covers five core dimensions: perception, reasoning, navigation, transformation, and configuration, comprising 17 fine-grained subtasks. Through careful manual annotation by experts with architectural backgrounds, we construct 3,000 question-answer pairs to enable comprehensive evaluation of architectural spatial intelligence. Based on ArchSIBench, we evaluate various VLMs and find that the architectural spatial intelligence of most models shows significant differences from human baselines; additionally, models exhibit substantial variability across capability dimensions. Some state-of-the-art models can approach the level of human evaluators without architectural training. However, a clear gap remains compared to human evaluators with architectural training, particularly in spatial transformation and configuration reasoning. We believe that ArchSIBench will provide important insights and systematic resources for measuring and advancing the architectural spatial intelligence of VLMs. The dataset and code are available at https://huggingface.co/datasets/ArchSIBench/ArchSIBench.

Related papers

• Spatial4D-Bench: A Versatile 4D Spatial Intelligence Benchmark [68.52969403388082]
  Humans naturally possess 4D spatial intelligence, supporting a broad spectrum of reasoning abilities.<n>To what extent can Multimodal Large Language Models (MLLMs) achieve human-level 4D spatial intelligence?<n>We present Spatial4D-Bench, a versatile 4D spatial intelligence benchmark designed to assess the 4D spatial reasoning abilities of MLLMs.
  arXiv  Detail & Related papers  (2025-12-31T19:56:51Z)
• Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarks [108.15756345836901]
  We provide a comprehensive review of multimodal spatial reasoning tasks with large models.<n>We review advances in embodied AI, including vision-language navigation and action models.<n>We consider emerging modalities such as audio and egocentric video, which contribute to novel spatial understanding through new sensors.
  arXiv  Detail & Related papers  (2025-10-29T17:55:43Z)
• ArchGPT: Understanding the World's Architectures with Large Multimodal Models [6.504675786709239]
  We present ArchGPT, a multimodal architectural visual question answering (VQA) model.<n>This pipeline yields Arch-300K, a domain-specialized dataset of approximately 315,000 image-question-answer triplets.
  arXiv  Detail & Related papers  (2025-09-25T07:49:43Z)
• How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspective [103.44502230776352]
  We present a systematic investigation of Visual Spatial Reasoning (VSR) in Vision-Language Models (VLMs)<n>We categorize spatial intelligence into three levels of capability, ie, basic perception, spatial understanding, spatial planning, and curate SIBench, a spatial intelligence benchmark encompassing nearly 20 open-source datasets across 23 task settings.
  arXiv  Detail & Related papers  (2025-09-23T12:00:14Z)
• Spatial Understanding from Videos: Structured Prompts Meet Simulation Data [89.77871049500546]
  We present a unified framework for enhancing 3D spatial reasoning in pre-trained vision-language models without modifying their architecture.<n>This framework combines SpatialMind, a structured prompting strategy that decomposes complex scenes and questions into interpretable reasoning steps, with ScanForgeQA, a scalable question-answering dataset built from diverse 3D simulation scenes.
  arXiv  Detail & Related papers  (2025-06-04T07:36:33Z)
• SITE: towards Spatial Intelligence Thorough Evaluation [121.1493852562597]
  Spatial intelligence (SI) represents a cognitive ability encompassing the visualization, manipulation, and reasoning about spatial relationships.<n>We introduce SITE, a benchmark dataset towards SI Thorough Evaluation.<n>Our approach to curating the benchmark combines a bottom-up survey about 31 existing datasets and a top-down strategy drawing upon three classification systems in cognitive science.
  arXiv  Detail & Related papers  (2025-05-08T17:45:44Z)
• Evaluation of Architectural Synthesis Using Generative AI [49.1574468325115]
  This paper presents a comparative evaluation of two systems: GPT-4o and Claude 3.5, in the task of architectural 3D synthesis.<n>We conduct a case study on two buildings from Palladio's Four Books of Architecture (1965): Villa Rotonda and Palazzo Porto.<n>We assess the systems' abilities in (1) interpreting 2D and 3D representations of buildings from drawings, (2) encoding the buildings into a CAD software script, and (3) self-improving based on outputs.
  arXiv  Detail & Related papers  (2025-03-04T18:39:28Z)
• Does Unsupervised Architecture Representation Learning Help Neural Architecture Search? [22.63641173256389]
  Existing Neural Architecture Search (NAS) methods either encode neural architectures using discrete encodings that do not scale well, or adopt supervised learning-based methods to jointly learn architecture representations and optimize architecture search on such representations which incurs search bias. We observe that the structural properties of neural architectures are hard to preserve in the latent space if architecture representation learning and search are coupled, resulting in less effective search performance.
  arXiv  Detail & Related papers  (2020-06-12T04:15:34Z)
• EgoMap: Projective mapping and structured egocentric memory for Deep RL [14.77454374243818]
  EgoMap is a spatially structured neural memory architecture. It augments a deep reinforcement learning agent's performance in 3D environments. We show it outperforms both standard recurrent agents and state of the art agents with structured memory.
  arXiv  Detail & Related papers  (2020-01-24T09:59:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.