AlloSpatial: Agentic Harness Framework for Spatial Reasoning in Foundation Models

• URL: http://arxiv.org/abs/2606.08952v1
• Date: Mon, 08 Jun 2026 02:55:05 GMT
• Title: AlloSpatial: Agentic Harness Framework for Spatial Reasoning in Foundation Models
• Abstract summary: AlloSpatial is an agentic framework for allocentric spatial cognition in foundation models.<n>World2Mind is a plug-and-play cognitive mapping sandbox that converts egocentric observations into structured allocentric priors.<n> experiments show that AlloSpatial improves proprietary models by 5%-18% in a training-free setting.
• Score: 33.84590534650041
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal Foundation Models (MFMs) have made substantial progress, yet remain fragile in spatial reasoning over the physical world. A key bottleneck lies in their inability to transform local egocentric observations into a global allocentric spatial representation. To address this, we propose AlloSpatial, an agentic framework for allocentric spatial cognition in foundation models. AlloSpatial introduces World2Mind, a plug-and-play cognitive mapping sandbox that converts egocentric observations into structured allocentric priors, including Allocentric-Spatial Trees and route maps that support querying object topology, geometric relations, passability, and trajectories. To utilize these priors reliably under noisy reconstruction and ambiguous visual evidence, AlloSpatial introduces a Spatial Reasoning Harness for tool-use judgment, modality-decoupled cue collection, and geometry-semantic arbitration. We further internalize this process in Qwen3-VL through cold-start reinforcement learning with a harness-gated trajectory-level reward. Experiments on VSI-Bench and MindCube show that AlloSpatial improves proprietary models by 5%-18% in a training-free setting, while ASTs alone support strong spatial reasoning even when visual inputs are removed. The trained AlloSpatial agents further outperform larger general-purpose models and competitive spatial baselines, suggesting that structured allocentric representations, active tool use, and verifiable reasoning offer a promising route toward spatially capable foundation models.

Related papers

• ChainSpace: A Chained-Reasoning Paradigm for Spatial Intelligence [13.06576024474319]
  We introduce ChainSpace, a chained-reasoning paradigm that structures spatial reasoning as a state-preserving multi-round process.<n>We instantiate ChainSpace-Bench, a manually annotated real-world benchmark with a Chain-Aware Metric, and ChainSpace-Pipeline, a simulator-based chain-structured supervision generation framework for spatial intelligence training.
  arXiv  Detail & Related papers  (2026-08-16T15:12:01Z)
• Trace, Verify, and Correct: A Training-Free Framework for Spatial Reasoning in Multimodal LLMs [11.359645786865359]
  We propose a modular and training-free framework for spatial reasoning verification and correction.<n>The framework associates atomic spatial evidence extracted from Chain-of-Thought reasoning with visual entities, spatial relations, source steps, and visual evidence.<n>Across 15 model-dataset settings, our method achieves an average accuracy of 68.94%, outperforming the compared baselines by 8.55 percentage points.
  arXiv  Detail & Related papers  (2026-08-05T12:26:23Z)
• SpaceEra++: A Unified Framework Towards 3D Spatial Reasoning in Video [88.79625979053873]
  We propose a novel framework, SpaceEra, for pre-trained vision-language models (VLMs)<n>We extend the original framework into a comprehensive system, termed SpaceEra++, which spans data construction, model design, training optimization, and prompting inference.<n>In addition, to enhance spatial reasoning, we develop SpaceAlign, which enforces pairwise object constraints by jointly exploiting absolute coordinates and relative spatial relations.
  arXiv  Detail & Related papers  (2026-07-02T06:56:29Z)
• S-Agent: Spatial Tool-Use Elicits Reasoning for Spatial Intelligence [77.5121349100339]
  textscS-Agent reshapes spatial perception into scene-centric understanding beyond frame-centric recognition.<n>textscS-Agent casts the VLM as a semantic planner that decides what evidence is needed.<n>textscS-Agent consistently improves both open-source and closed-source VLMs in a training-free manner.
  arXiv  Detail & Related papers  (2026-06-18T17:34:55Z)
• Self-Evolving Spatial Reasoning in Vision Language Models via Geometric Logic Consistency [53.28076739140119]
  We propose a framework that enforces logical consistency in Vision-Language Models (VLMs) through geometric and linguistic duality operations.<n>SAGE is model-agnostic, data-efficient compared to prior GRPO methods, and can be applied as a lightweight post-training stage to any existing VLM.<n> Experiments on video and spatial reasoning benchmarks demonstrate consistent improvements over strong baselines.
  arXiv  Detail & Related papers  (2026-05-18T10:05:21Z)
• SpatialImaginer: Towards Adaptive Visual Imagination for Spatial Reasoning [67.67774742200626]
  spatial intelligence, which refers to the ability to reason about geometric and physical structure from visual observations, is a core challenge for large language models.<n>We propose a unified multimodal generation framework that integrates textual reasoning with visual imagination.<n>Our framework adopts a divide-and-conquer strategy, using text chain-of-thought for high-level semantic planning and the visual imagination for geometry-sensitive state transformation and consistency preservation.
  arXiv  Detail & Related papers  (2026-04-19T11:21:59Z)
• SpatialReward: Verifiable Spatial Reward Modeling for Fine-Grained Spatial Consistency in Text-to-Image Generation [62.55421542903781]
  We present textbfSpatialReward, a verifiable reward model explicitly designed to evaluate spatial layouts in generated images.<n>Experiments on Stable Diffusion and FLUX show that incorporating SpatialReward into RL training consistently improves spatial consistency and overall generation quality.
  arXiv  Detail & Related papers  (2026-03-23T17:26:35Z)
• GeoSense: Internalizing Geometric Necessity Perception for Multimodal Reasoning [51.63457948949102]
  We develop a framework to overcome the limited spatial understanding of Multimodal Large Language Models (MLLMs)<n>Our framework endows the model with an awareness of perceptual insufficiency, empowering it to autonomously engage geometric features in reasoning when 2D cues are deemed insufficient.
  arXiv  Detail & Related papers  (2026-03-11T03:32:12Z)
• World2Mind: Cognition Toolkit for Allocentric Spatial Reasoning in Foundation Models [25.135316296678187]
  Existing methods either overfit statistical shortcuts via 3D grounding data or remain confined to 2D visual perception.<n>We propose World2Mind, a training-free spatial intelligence toolkit.<n>We show that World2Mind boosts the performance of frontier models, such as GPT-5.2, by 5%18%.
  arXiv  Detail & Related papers  (2026-03-10T15:12:14Z)
• 3ViewSense: Spatial and Mental Perspective Reasoning from Orthographic Views in Vision-Language Models [16.924616915709123]
  Vision-Language Models fail to construct coherent 3D mental representations from 2D observations.<n>We introduce textbf3ViewSense', a framework that grounds spatial reasoning in Orthographic Views.<n> Empirical results on spatial reasoning benchmarks demonstrate that our method significantly outperforms existing baselines.
  arXiv  Detail & Related papers  (2026-03-08T17:57:56Z)
• Thinking with Geometry: Active Geometry Integration for Spatial Reasoning [68.59084007360615]
  We propose GeoThinker, a framework that shifts paradigm passive fusion to active perception.<n>Instead of feature mixing, GeoThinker enables the model to selectively retrieve geometric evidence conditioned on its internal reasoning demands.<n>Our results indicate that the ability to actively integrate spatial structures is essential for next-generation spatial intelligence.
  arXiv  Detail & Related papers  (2026-02-05T18:59:32Z)
• Allocentric Perceiver: Disentangling Allocentric Reasoning from Egocentric Visual Priors via Frame Instantiation [41.434638833165494]
  Allocentric Perceiver is a training-free strategy that recovers metric 3D states from one or more images with off-the-shelf geometric experts.<n>Allocentric Perceriver offloads mental rotation from implicit reasoning to explicit computation.
  arXiv  Detail & Related papers  (2026-02-05T15:45:39Z)
• EagleVision: A Dual-Stage Framework with BEV-grounding-based Chain-of-Thought for Spatial Intelligence [10.889641815961133]
  spatial intelligence approaches typically attach 3D cues to 2D reasoning pipelines or MLLMs with black-box reconstruction modules.<n>We present EagleVision, a framework for progressive spatial cognition through macro perception and micro verification.
  arXiv  Detail & Related papers  (2025-12-17T07:51:36Z)
• SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Models [73.19077622773075]
  We present a comprehensive methodology for building spatial intelligence progressively.<n>We introduce SpatialLadder-26k, a multimodal dataset containing 26,610 samples spanning object localization, single image, multi-view, and video spatial reasoning tasks.<n>We design a three-stage progressive training framework that establishes spatial perception through object localization, develops spatial understanding through multi-dimensional spatial tasks, and strengthens complex reasoning via reinforcement learning with verifiable rewards.
  arXiv  Detail & Related papers  (2025-10-09T17:50:54Z)
• SPHERE: Unveiling Spatial Blind Spots in Vision-Language Models Through Hierarchical Evaluation [7.659514491338669]
  Current vision-language models may grasp basic spatial cues but struggle with the multi-dimensional spatial reasoning necessary for human-like understanding and real-world applications.<n>We develop SPHERE, a hierarchical evaluation framework supported by a new human-annotated dataset.<n> Benchmark evaluation of state-of-the-art models reveals significant deficiencies, especially in reasoning about distance and proximity.
  arXiv  Detail & Related papers  (2024-12-17T09:10:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.