GeoSense: Internalizing Geometric Necessity Perception for Multimodal Reasoning

• URL: http://arxiv.org/abs/2603.10370v1
• Date: Wed, 11 Mar 2026 03:32:12 GMT
• Title: GeoSense: Internalizing Geometric Necessity Perception for Multimodal Reasoning
• Abstract summary: We develop a framework to overcome the limited spatial understanding of Multimodal Large Language Models (MLLMs)<n>Our framework endows the model with an awareness of perceptual insufficiency, empowering it to autonomously engage geometric features in reasoning when 2D cues are deemed insufficient.
• Score: 51.63457948949102
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Advancing towards artificial superintelligence requires rich and intelligent perceptual capabilities. A critical frontier in this pursuit is overcoming the limited spatial understanding of Multimodal Large Language Models (MLLMs), where geometry information is essential. Existing methods often address this by rigidly injecting geometric signals into every input, while ignoring their necessity and adding computation overhead. Contrary to this paradigm, our framework endows the model with an awareness of perceptual insufficiency, empowering it to autonomously engage geometric features in reasoning when 2D cues are deemed insufficient. To achieve this, we first introduce an independent geometry input channel to the model architecture and conduct alignment training, enabling the effective utilization of geometric features. Subsequently, to endow the model with perceptual awareness, we curate a dedicated spatial-aware supervised fine-tuning dataset. This serves to activate the model's latent internal cues, empowering it to autonomously determine the necessity of geometric information. Experiments across multiple spatial reasoning benchmarks validate this approach, demonstrating significant spatial gains without compromising 2D visual reasoning capabilities, offering a path toward more robust, efficient and self-aware multi-modal intelligence.

Related papers

• Geometry-Aware Reinforcement Learning for 2D Irregular Nesting [0.0]
  We argue that Reinforcement Learning is uniquely positioned to overcome this bottleneck.<n>By pairing an optimization policy with a geometry-aware neural encoder, an agent can automatically discover rich geometric priors.<n>We show that our trained agent area utilization performance is highly competitive with Sparrow, the state-of-the-art solver.
  arXiv  Detail & Related papers  (2026-06-09T09:11:36Z)
• Learning Geometric Representations from Videos for Spatial Intelligent Multimodal Large Language Models [32.477423778941386]
  We present GeoVR, a framework that learns geometric representations using purely 2D video sequences.<n>Rather than employing superficial feature mixing, GeoVR reshapes the internal representations of the MLLM by distilling geometry knowledge.<n>Experiments on spatial reasoning benchmarks demonstrate that GeoVR achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-06-04T08:11:12Z)
• From Where Things Are to What They Are For: Benchmarking Spatial-Functional Intelligence in Multimodal LLMs [19.943841049221625]
  We introduce the Spatial-Functional Intelligence Benchmark (SFI-Bench), a video-based benchmark with over 1,500 expert-annotated questions.<n>SFI-Bench systematically evaluates two complementary dimensions of advanced reasoning.
  arXiv  Detail & Related papers  (2026-05-04T01:19:36Z)
• SpatialImaginer: Towards Adaptive Visual Imagination for Spatial Reasoning [67.67774742200626]
  spatial intelligence, which refers to the ability to reason about geometric and physical structure from visual observations, is a core challenge for large language models.<n>We propose a unified multimodal generation framework that integrates textual reasoning with visual imagination.<n>Our framework adopts a divide-and-conquer strategy, using text chain-of-thought for high-level semantic planning and the visual imagination for geometry-sensitive state transformation and consistency preservation.
  arXiv  Detail & Related papers  (2026-04-19T11:21:59Z)
• SpatialEvo: Self-Evolving Spatial Intelligence via Deterministic Geometric Environments [75.60795462502949]
  SpatialEvo is a self-evolving framework for 3D spatial reasoning.<n>It formalizes 16 spatial reasoning task categories under explicit geometric validation rules.<n>It converts unannotated 3D scenes into zero-noise interactive oracles, replacing model consensus with objective physical feedback.
  arXiv  Detail & Related papers  (2026-04-15T17:59:12Z)
• World2Mind: Cognition Toolkit for Allocentric Spatial Reasoning in Foundation Models [25.135316296678187]
  Existing methods either overfit statistical shortcuts via 3D grounding data or remain confined to 2D visual perception.<n>We propose World2Mind, a training-free spatial intelligence toolkit.<n>We show that World2Mind boosts the performance of frontier models, such as GPT-5.2, by 5%18%.
  arXiv  Detail & Related papers  (2026-03-10T15:12:14Z)
• Thinking with Geometry: Active Geometry Integration for Spatial Reasoning [68.59084007360615]
  We propose GeoThinker, a framework that shifts paradigm passive fusion to active perception.<n>Instead of feature mixing, GeoThinker enables the model to selectively retrieve geometric evidence conditioned on its internal reasoning demands.<n>Our results indicate that the ability to actively integrate spatial structures is essential for next-generation spatial intelligence.
  arXiv  Detail & Related papers  (2026-02-05T18:59:32Z)
• Forging Spatial Intelligence: A Roadmap of Multi-Modal Data Pre-Training for Autonomous Systems [75.78934957242403]
  Self-driving vehicles and drones require true Spatial Intelligence from multi-modal onboard sensor data.<n>This paper presents a framework for multi-modal pre-training, identifying the core set of techniques driving progress toward this goal.
  arXiv  Detail & Related papers  (2025-12-30T17:58:01Z)
• COOPER: A Unified Model for Cooperative Perception and Reasoning in Spatial Intelligence [57.63155257058967]
  We investigate whether a unified MLLM can develop an intrinsic ability to enhance spatial perception and, through adaptive interleaved reasoning, achieve stronger spatial intelligence.<n>We propose textbfCOOPER, a unified MLLM that leverages depth and segmentation as auxiliary modalities and is trained in two stages to acquire auxiliary modality generation and adaptive, interleaved reasoning capabilities.
  arXiv  Detail & Related papers  (2025-12-04T08:26:04Z)
• Cross-Modal Geometric Hierarchy Fusion: An Implicit-Submap Driven Framework for Resilient 3D Place Recognition [9.411542547451193]
  We propose a novel framework that redefines 3D place recognition through density-agnostic geometric reasoning.<n>Specifically, we introduce an implicit 3D representation based on elastic points, which is immune to the interference of original scene point cloud density.<n>With the aid of these two types of information, we obtain descriptors that fuse geometric information from both bird's-eye view and 3D segment perspectives.
  arXiv  Detail & Related papers  (2025-06-17T07:04:07Z)
• Spatial Knowledge Graph-Guided Multimodal Synthesis [78.11669780958657]
  We introduce a novel multimodal synthesis approach guided by spatial knowledge graphs, grounded in the concept of knowledge-to-data generation.<n>In experiments, data synthesized from diverse types of spatial knowledge, including direction and distance, enhance the spatial perception and reasoning abilities of MLLMs markedly.<n>We hope that the idea of knowledge-based data synthesis can advance the development of spatial intelligence.
  arXiv  Detail & Related papers  (2025-05-28T17:50:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.