Make Geometry Matter for Spatial Reasoning

• URL: http://arxiv.org/abs/2603.26639v1
• Date: Fri, 27 Mar 2026 17:45:12 GMT
• Title: Make Geometry Matter for Spatial Reasoning
• Abstract summary: Vision-language models (VLMs) achieve strong image and video understanding, yet their ability to perform spatial reasoning in both static scenes and dynamic videos remains limited.<n>Recent advances try to handle this limitation by injecting geometry tokens from pretrained 3D foundation models into VLMs.<n>We propose GeoSR, a framework designed to make geometry matter by encouraging VLMs to actively reason with geometry tokens.
• Score: 62.61667611352403
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Empowered by large-scale training, vision-language models (VLMs) achieve strong image and video understanding, yet their ability to perform spatial reasoning in both static scenes and dynamic videos remains limited. Recent advances try to handle this limitation by injecting geometry tokens from pretrained 3D foundation models into VLMs. Nevertheless, we observe that naive token fusion followed by standard fine-tuning in this line of work often leaves such geometric cues underutilized for spatial reasoning, as VLMs tend to rely heavily on 2D visual cues. In this paper, we propose GeoSR, a framework designed to make geometry matter by encouraging VLMs to actively reason with geometry tokens. GeoSR introduces two key components: (1) Geometry-Unleashing Masking, which strategically masks portions of 2D vision tokens during training to weaken non-geometric shortcuts and force the model to consult geometry tokens for spatial reasoning; and (2) Geometry-Guided Fusion, a gated routing mechanism that adaptively amplifies geometry token contributions in regions where geometric evidence is critical. Together, these designs unleash the potential of geometry tokens for spatial reasoning tasks. Extensive experiments on both static and dynamic spatial reasoning benchmarks demonstrate that GeoSR consistently outperforms prior methods and establishes new state-of-the-art performance by effectively leveraging geometric information. The project page is available at https://suhzhang.github.io/GeoSR/.

Related papers

• Learning Geometric Representations from Videos for Spatial Intelligent Multimodal Large Language Models [32.477423778941386]
  We present GeoVR, a framework that learns geometric representations using purely 2D video sequences.<n>Rather than employing superficial feature mixing, GeoVR reshapes the internal representations of the MLLM by distilling geometry knowledge.<n>Experiments on spatial reasoning benchmarks demonstrate that GeoVR achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-06-04T08:11:12Z)
• GeoWeaver: Grounding Visual Tokens with Geometric Evidence before Scene Reasoning [45.229974852899716]
  multimodal models incorporate geometric information branches, 3D-aware supervision, reasoning-stage fusion, or long-horizon memory.<n>These approaches typically treat geometric cues as a shared signal across all visual tokens.<n>We introduce GeoWeaver, a framework that treats geometry as a representational prerequisite fortemporal reasoning.
  arXiv  Detail & Related papers  (2026-05-21T14:40:03Z)
• GeoAlign: Geometric Feature Realignment for MLLM Spatial Reasoning [12.641199394796436]
  Multimodal large language models (MLLMs) have exhibited remarkable performance in various visual tasks, yet still struggle with spatial reasoning.<n>Recent efforts mitigate this by injecting geometric features from 3D foundation models, but rely on static single-layer extractions.<n>We propose GeoAlign, a novel framework that dynamically aggregates multi-layer geometric features to realign with the actual demands.
  arXiv  Detail & Related papers  (2026-04-14T11:58:02Z)
• Learning 3D Representations for Spatial Intelligence from Unposed Multi-View Images [81.94999489820974]
  UniSplat is a feed-forward framework for learning 3D representations from unposed multi-view images.<n>We introduce a dual-masking strategy that strengthens geometry induction in the encoder.<n>Second, we develop a coarse-to-fine Gaussian splatting strategy that reduces appearance-semantics inconsistencies.<n>Third, we introduce a pose-conditioned recalibration mechanism that interrelates predicted 3D point and semantic maps into the image plane.
  arXiv  Detail & Related papers  (2026-04-12T10:36:18Z)
• Let Geometry GUIDE: Layer-wise Unrolling of Geometric Priors in Multimodal LLMs [13.627465963609936]
  GUIDE (Geometric Unrolling Inside MLLM Early-layers) is a progressive geometric priors injection framework.<n>We introduce a context-aware gating that enables the model to fetch requisite spatial cues based on current semantics.
  arXiv  Detail & Related papers  (2026-04-07T10:45:28Z)
• Scalable Adaptation of 3D Geometric Foundation Models via Weak Supervision from Internet Video [76.32954467706581]
  We propose SAGE, a framework for Scalable Adaptation of GEometric foundation models from raw video streams.<n>We use a hierarchical mining pipeline to transform videos into training trajectories and hybrid supervision.<n>Experiments show that SAGE significantly enhances zero-shot generalization, reducing Chamfer Distance by 20-42% on unseen benchmarks.
  arXiv  Detail & Related papers  (2026-02-08T09:53:21Z)
• Thinking with Geometry: Active Geometry Integration for Spatial Reasoning [68.59084007360615]
  We propose GeoThinker, a framework that shifts paradigm passive fusion to active perception.<n>Instead of feature mixing, GeoThinker enables the model to selectively retrieve geometric evidence conditioned on its internal reasoning demands.<n>Our results indicate that the ability to actively integrate spatial structures is essential for next-generation spatial intelligence.
  arXiv  Detail & Related papers  (2026-02-05T18:59:32Z)
• Learning to Reason in 4D: Dynamic Spatial Understanding for Vision Language Models [79.18306680174011]
  DSR Suite bridges gap across aspects of dataset, benchmark and model.<n>We propose an automated pipeline that generates multiple-choice question-answer pairs from in-the-wild videos for DSR.<n>The pipeline extracts rich geometric and motion information, including camera poses, local point clouds, object masks, orientations, and 3D trajectories.
  arXiv  Detail & Related papers  (2025-12-23T17:56:36Z)
• GeoWorld: Unlocking the Potential of Geometry Models to Facilitate High-Fidelity 3D Scene Generation [68.02988074681427]
  Previous works leveraging video models for image-to-3D scene generation tend to suffer from geometric distortions and blurry content.<n>In this paper, we renovate the pipeline of image-to-3D scene generation by unlocking the potential of geometry models.<n>Our GeoWorld can generate high-fidelity 3D scenes from a single image and a given camera trajectory, outperforming prior methods both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2025-11-28T13:55:45Z)
• DynaSolidGeo: A Dynamic Benchmark for Genuine Spatial Mathematical Reasoning of VLMs in Solid Geometry [21.08408074777344]
  DynaSolidGeo is a benchmark for evaluating genuine spatial reasoning in Vision-Language Models (VLMs)<n>It contains 503 expert-curated seed questions that can, in principle, dynamically generate an unbounded number of diverse multimodal text-visual instances.<n>We incorporate process evaluation based on expert-annotated reasoning chains to measure logical validity and causal coherence.
  arXiv  Detail & Related papers  (2025-10-25T15:49:45Z)
• GACO-CAD: Geometry-Augmented and Conciseness-Optimized CAD Model Generation from Single Image [11.612167656421079]
  Multi-modal large language models (MLLMs) still struggle with accurately inferring 3D geometry from 2D images.<n>We introduce GACO-CAD, a novel two-stage post-training framework.<n>Experiments on the DeepCAD and Fusion360 datasets show that GACO-CAD achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-10-20T04:57:20Z)
• Geometry-guided Feature Learning and Fusion for Indoor Scene Reconstruction [14.225228781008209]
  This paper proposes a novel geometry integration mechanism for 3D scene reconstruction. Our approach incorporates 3D geometry at three levels, i.e. feature learning, feature fusion, and network supervision.
  arXiv  Detail & Related papers  (2024-08-28T08:02:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.