Spatial Reasoning is Not a Free Lunch: A Controlled Study on LLaVA

• URL: http://arxiv.org/abs/2603.12545v1
• Date: Fri, 13 Mar 2026 01:11:23 GMT
• Title: Spatial Reasoning is Not a Free Lunch: A Controlled Study on LLaVA
• Abstract summary: Vision-language models (VLMs) have advanced rapidly, yet they still struggle with basic spatial reasoning.<n>We argue that this failure is not merely a data problem, but is closely tied to dominant design choices in current VLM pipelines.<n>We present a controlled diagnostic study within the LLaVA framework to isolate how these choices affect spatial grounding.
• Score: 3.9393480686002715
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-language models (VLMs) have advanced rapidly, yet they still struggle with basic spatial reasoning. Despite strong performance on general benchmarks, modern VLMs remain brittle at understanding 2D spatial relationships such as relative position, layout, and counting. We argue that this failure is not merely a data problem, but is closely tied to dominant design choices in current VLM pipelines: reliance on CLIP-style image encoders and the flattening of images into 1D token sequences with 1D positional encoding. We present a controlled diagnostic study within the LLaVA framework to isolate how these choices affect spatial grounding. We evaluate frontier models and LLaVA variants on a suite of spatial benchmarks, comparing CLIP-based encoders against alternatives trained with denser or generative objectives, as well as variants augmented with 2D positional encoding. Our results show consistent spatial performance gaps across models, and indicate that encoder objectives and positional structure shape spatial behavior, but do not fully resolve it.

Related papers

• Embodied-Navigator: Point, Think, Memorize, and Align for Efficient Navigation [68.02899606570223]
  We propose TAMP-Nav, a unified framework for efficient embodied navigation.<n>First, we introduce a Pixel-to-3D Action Formulation (Point) that reformulates navigation into 2D visual coordinates.<n>Second, we propose an integrated Selective Reasoning and Anchor-Trajectory Memory mechanism (Think and Memorize)
  arXiv  Detail & Related papers  (2026-08-18T08:37:08Z)
• Mitigating Positional Leakage in 3D Masked Autoencoders for Robust Representation Learning [36.571886409536255]
  Masked autoencoding has emerged as a prominent paradigm for self-supervised learning on 3D point clouds.<n>In this work, we identify that the decoder in existing 3D MAE frameworks tends to over-rely on positional information.<n>We propose MPL-MAE, a masked point learning framework that mitigates positional over-reliance while enhancing the utilization of encoder features.
  arXiv  Detail & Related papers  (2026-06-30T12:29:34Z)
• SPACENUM: Revisiting Spatial Numerical Understanding in VLMs [14.025058020258816]
  We study whether Vision-Language Models (VLMs) truly understand numerical values in spatial settings.<n>Across dynamic transitions and static layouts, we find that models largely fail to ground numbers in spatial meaning.<n>We show that explicit reasoning provides only marginal gains, while tuning can partially improve spatial numerical understanding.
  arXiv  Detail & Related papers  (2026-05-22T17:58:36Z)
• VEGA: Visual Encoder Grounding Alignment for Spatially-Aware Vision-Language-Action Models [55.12929235609365]
  Visual backbones of current vision-language-action (VLA) models are predominantly pretrained on 2D image data without explicit 3D geometric supervision.<n>Existing implicit grounding methods partially address this by aligning VLA features with those of 3D-aware foundation models.<n>We propose VEGA, a framework that directly aligns the output of the VLA's visual encoder with spatially-aware features from DINOv2-FiT3D.
  arXiv  Detail & Related papers  (2026-05-11T12:44:26Z)
• The Spatial Blindspot of Vision-Language Models [3.9393480686002715]
  Vision-language models (VLMs) have advanced rapidly, but their ability to capture spatial relationships remains a blindspot.<n>We argue that this lack of spatial awareness is a missing dimension in VLM design and a bottleneck for applications requiring spatial grounding.
  arXiv  Detail & Related papers  (2026-01-15T00:30:34Z)
• SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Driving [38.21244888074097]
  SpaceDrive is a spatial-aware driving framework that treats spatial information as explicit positional encodings (PEs) instead of textual digit tokens.<n>We show that SpaceDrive achieves state-of-the-art open-loop performance on the nuScenes dataset and the second-best Driving Score of 78.02 on the Bench2Drive benchmark.
  arXiv  Detail & Related papers  (2025-12-11T14:59:07Z)
• SpatialGeo:Boosting Spatial Reasoning in Multimodal LLMs via Geometry-Semantics Fusion [23.86761713752287]
  Multimodal large language models (MLLMs) have achieved significant progress in image and language tasks.<n>Most MLLMs suffer from limited spatial reasoning ability to interpret and infer spatial arrangements in three-dimensional space.<n>We propose a novel vision encoder based on hierarchical fusion of geometry and semantics features, generating spatial-aware visual embedding.
  arXiv  Detail & Related papers  (2025-11-21T15:24:33Z)
• Spatial Reasoning in Foundation Models: Benchmarking Object-Centric Spatial Understanding [8.202861909913791]
  We present a benchmark for object-centric spatial reasoning in foundation models.<n>We find a stable trade-off: detectors such as GroundingDINO and OWLv2 deliver precise boxes with limited relational reasoning.<n>Our study highlights the gap between localization and true spatial understanding, and pointing toward the need for spatially-aware foundation models in the community.
  arXiv  Detail & Related papers  (2025-09-26T06:06:19Z)
• SD-VLM: Spatial Measuring and Understanding with Depth-Encoded Vision-Language Models [75.64836077468722]
  Vision language models (VLMs) excel in 2D semantic visual understanding, but their ability to quantitatively reason about 3D spatial relationships remains under-explored.<n>We propose SD-VLM, a novel framework that significantly enhances fundamental spatial perception abilities of VLMs.<n>We have trained SD-VLM, a strong generalist VLM which shows superior quantitative spatial measuring and understanding capability.
  arXiv  Detail & Related papers  (2025-09-22T12:08:12Z)
• FloorplanQA: A Benchmark for Spatial Reasoning in LLMs using Structured Representations [78.65988445433844]
  FloorplanQA is a diagnostic benchmark for evaluating spatial reasoning in large-language models.<n>The benchmark covers core spatial tasks, including distance measurement, visibility, path finding, and object placement within constrained spaces.
  arXiv  Detail & Related papers  (2025-07-10T11:16:48Z)
• Complexity Matters: Rethinking the Latent Space for Generative Modeling [65.64763873078114]
  In generative modeling, numerous successful approaches leverage a low-dimensional latent space, e.g., Stable Diffusion. In this study, we aim to shed light on this under-explored topic by rethinking the latent space from the perspective of model complexity.
  arXiv  Detail & Related papers  (2023-07-17T07:12:29Z)
• On Robust Cross-View Consistency in Self-Supervised Monocular Depth Estimation [56.97699793236174]
  We study two kinds of robust cross-view consistency in this paper. We exploit the temporal coherence in both depth feature space and 3D voxel space for self-supervised monocular depth estimation. Experimental results on several outdoor benchmarks show that our method outperforms current state-of-the-art techniques.
  arXiv  Detail & Related papers  (2022-09-19T03:46:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.