3ViewSense: Spatial and Mental Perspective Reasoning from Orthographic Views in Vision-Language Models

• URL: http://arxiv.org/abs/2603.07751v1
• Date: Sun, 08 Mar 2026 17:57:56 GMT
• Title: 3ViewSense: Spatial and Mental Perspective Reasoning from Orthographic Views in Vision-Language Models
• Abstract summary: Vision-Language Models fail to construct coherent 3D mental representations from 2D observations.<n>We introduce textbf3ViewSense', a framework that grounds spatial reasoning in Orthographic Views.<n> Empirical results on spatial reasoning benchmarks demonstrate that our method significantly outperforms existing baselines.
• Score: 16.924616915709123
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Current Large Language Models have achieved Olympiad-level logic, yet Vision-Language Models paradoxically falter on elementary spatial tasks like block counting. This capability mismatch reveals a critical ``spatial intelligence gap,'' where models fail to construct coherent 3D mental representations from 2D observations. We uncover this gap via diagnostic analyses showing the bottleneck is a missing view-consistent spatial interface rather than insufficient visual features or weak reasoning. To bridge this, we introduce \textbf{3ViewSense}, a framework that grounds spatial reasoning in Orthographic Views. Drawing on engineering cognition, we propose a ``Simulate-and-Reason'' mechanism that decomposes complex scenes into canonical orthographic projections to resolve geometric ambiguities. By aligning egocentric perceptions with these allocentric references, our method facilitates explicit mental rotation and reconstruction. Empirical results on spatial reasoning benchmarks demonstrate that our method significantly outperforms existing baselines, with consistent gains on occlusion-heavy counting and view-consistent spatial reasoning. The framework also improves the stability and consistency of spatial descriptions, offering a scalable path toward stronger spatial intelligence in multimodal systems.

Related papers

• Disentangling 3D Modeling from Spatial Reasoning [56.3819096070689]
  We explore an alternative paradigm for spatial reasoning by explicitly disentangling 3D perception from reasoning.<n>Our key observation is that modern perception models excel at estimating continuous 3D geometry, whereas large language models (LLMs) are particularly effective at compositional and symbolic reasoning.<n>Motivated by these complementary strengths, we propose the Disentangled Spatial Reasoner (DiSR)<n>DiSR reconstructs the physical world into structured 3D evidence using off-the-shelf expert perception models and fine-tunes an LLM with LoRA to perform reasoning solely over this explicit geometric evidence.
  arXiv  Detail & Related papers  (2026-08-05T14:32:48Z)
• SpaceEra++: A Unified Framework Towards 3D Spatial Reasoning in Video [88.79625979053873]
  We propose a novel framework, SpaceEra, for pre-trained vision-language models (VLMs)<n>We extend the original framework into a comprehensive system, termed SpaceEra++, which spans data construction, model design, training optimization, and prompting inference.<n>In addition, to enhance spatial reasoning, we develop SpaceAlign, which enforces pairwise object constraints by jointly exploiting absolute coordinates and relative spatial relations.
  arXiv  Detail & Related papers  (2026-07-02T06:56:29Z)
• SpatialImaginer: Towards Adaptive Visual Imagination for Spatial Reasoning [67.67774742200626]
  spatial intelligence, which refers to the ability to reason about geometric and physical structure from visual observations, is a core challenge for large language models.<n>We propose a unified multimodal generation framework that integrates textual reasoning with visual imagination.<n>Our framework adopts a divide-and-conquer strategy, using text chain-of-thought for high-level semantic planning and the visual imagination for geometry-sensitive state transformation and consistency preservation.
  arXiv  Detail & Related papers  (2026-04-19T11:21:59Z)
• Keep it SymPL: Symbolic Projective Layout for Allocentric Spatial Reasoning in Vision-Language Models [5.961445903498366]
  We introduce Projective Layout (SymPL), a framework that reformulates allocentric reasoning into symbolic forms that VLMs handle well.<n>Experiments demonstrate that this reformulation substantially improves performance in both allocentric and egocentric tasks.
  arXiv  Detail & Related papers  (2026-02-22T10:18:54Z)
• Allocentric Perceiver: Disentangling Allocentric Reasoning from Egocentric Visual Priors via Frame Instantiation [41.434638833165494]
  Allocentric Perceiver is a training-free strategy that recovers metric 3D states from one or more images with off-the-shelf geometric experts.<n>Allocentric Perceriver offloads mental rotation from implicit reasoning to explicit computation.
  arXiv  Detail & Related papers  (2026-02-05T15:45:39Z)
• Thinking with Blueprints: Assisting Vision-Language Models in Spatial Reasoning via Structured Object Representation [52.605647992080485]
  spatial reasoning advances vision-language models from visual perception toward semantic understanding.<n>We integrate the cognitive concept of an object-centric blueprint into spatial reasoning.<n>Our method consistently outperforms existing vision-language models.
  arXiv  Detail & Related papers  (2026-01-05T10:38:26Z)
• Seeing through Imagination: Learning Scene Geometry via Implicit Spatial World Modeling [68.14113731953971]
  This paper introduces MILO, an Implicit spatIaL wOrld modeling paradigm that simulates human-like imagination.<n>We show that our approach significantly enhances spatial reasoning capabilities across multiple baselines and benchmarks.
  arXiv  Detail & Related papers  (2025-12-01T16:01:41Z)
• Enhancing Spatial Reasoning in Multimodal Large Language Models through Reasoning-based Segmentation [50.81551581148339]
  We introduce Relevant Reasoning (R$2$S), a reasoning-based segmentation framework.<n>We also introduce 3D ReasonSeg, a reasoning-based segmentation dataset.<n>Both experiments demonstrate that the R$2$S and 3D ReasonSeg effectively endow 3D point cloud perception with stronger spatial reasoning capabilities.
  arXiv  Detail & Related papers  (2025-06-29T06:58:08Z)
• Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models [14.442394137843923]
  We present a detailed analysis that first delineates the core elements of spatial reasoning.<n>We then assesses the performance of these models in both synthetic and real-world images.
  arXiv  Detail & Related papers  (2025-03-25T14:34:06Z)
• Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas [69.56484419619919]
  We study the spatial reasoning challenge from the lens of mechanistic interpretability.<n>We observe that successful spatial reasoning correlates strongly with the model's ability to align its attention with actual object locations.<n>Motivated by these findings, we propose ADAPTVIS to sharpen the attention on highly relevant regions when confident.
  arXiv  Detail & Related papers  (2025-03-03T17:57:03Z)
• SPHERE: Unveiling Spatial Blind Spots in Vision-Language Models Through Hierarchical Evaluation [7.659514491338669]
  Current vision-language models may grasp basic spatial cues but struggle with the multi-dimensional spatial reasoning necessary for human-like understanding and real-world applications.<n>We develop SPHERE, a hierarchical evaluation framework supported by a new human-annotated dataset.<n> Benchmark evaluation of state-of-the-art models reveals significant deficiencies, especially in reasoning about distance and proximity.
  arXiv  Detail & Related papers  (2024-12-17T09:10:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.