Visual Spatial Tuning

• URL: http://arxiv.org/abs/2511.05491v1
• Date: Fri, 07 Nov 2025 18:59:16 GMT
• Title: Visual Spatial Tuning
• Authors: Rui Yang, Ziyu Zhu, Yanwei Li, Jingjia Huang, Shen Yan, Siyuan Zhou, Zhe Liu, Xiangtai Li, Shuangye Li, Wenqian Wang, Yi Lin, Hengshuang Zhao,
• Abstract summary: We introduce Visual Spatial Tuning (VST), a comprehensive framework to cultivate Vision-Language Models (VLMs) with human-like visuospatial abilities.<n>In particular, we adopt a progressive training pipeline: supervised fine-tuning to build foundational spatial knowledge, followed by reinforcement learning to further improve spatial reasoning abilities.<n>It turns out that the Vision-Language-Action models can be significantly enhanced with proposed spatial tuning paradigm, paving the way for more physically grounded AI.
• Score: 98.96397681943255
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Capturing spatial relationships from visual inputs is a cornerstone of human-like general intelligence. Several previous studies have tried to enhance the spatial awareness of Vision-Language Models (VLMs) by adding extra expert encoders, which brings extra overhead and usually harms general capabilities. To enhance the spatial ability in general architectures, we introduce Visual Spatial Tuning (VST), a comprehensive framework to cultivate VLMs with human-like visuospatial abilities, from spatial perception to reasoning. We first attempt to enhance spatial perception in VLMs by constructing a large-scale dataset termed VST-P, which comprises 4.1 million samples spanning 19 skills across single views, multiple images, and videos. Then, we present VST-R, a curated dataset with 135K samples that instruct models to reason in space. In particular, we adopt a progressive training pipeline: supervised fine-tuning to build foundational spatial knowledge, followed by reinforcement learning to further improve spatial reasoning abilities. Without the side-effect to general capabilities, the proposed VST consistently achieves state-of-the-art results on several spatial benchmarks, including $34.8\%$ on MMSI-Bench and $61.2\%$ on VSIBench. It turns out that the Vision-Language-Action models can be significantly enhanced with the proposed spatial tuning paradigm, paving the way for more physically grounded AI.

Related papers

• Cambrian-S: Towards Spatial Supersensing in Video [78.46305169769884]
  We frame spatial supersensing as four stages beyond linguistic-only understanding: semantic perception, streaming event cognition, implicit 3D spatial cognition, and predictive world modeling.<n>To drive progress in spatial supersensing, we present-SUPER, a two-part benchmark: VSR (long-horizon visual spatial recall) and VSC (continual visual spatial counting)<n>We then test data scaling limits by curating-590K and training Cambrian-S, achieving +30% absolute improvement on arbitrarily general capabilities.<n>We propose predictive sensing as a path forward, presenting a proof-of-concept in which a self-supervised
  arXiv  Detail & Related papers  (2025-11-06T18:55:17Z)
• SD-VLM: Spatial Measuring and Understanding with Depth-Encoded Vision-Language Models [75.64836077468722]
  Vision language models (VLMs) excel in 2D semantic visual understanding, but their ability to quantitatively reason about 3D spatial relationships remains under-explored.<n>We propose SD-VLM, a novel framework that significantly enhances fundamental spatial perception abilities of VLMs.<n>We have trained SD-VLM, a strong generalist VLM which shows superior quantitative spatial measuring and understanding capability.
  arXiv  Detail & Related papers  (2025-09-22T12:08:12Z)
• See&Trek: Training-Free Spatial Prompting for Multimodal Large Language Model [33.18304419115947]
  We introduce SEE&TREK, the first training-free prompting framework to enhance the spatial understanding of Multimodal Large Language Models (MLLM) under vision-only constraints.<n>We focus on increasing visual diversity and motion reconstruction.<n>Our method is training&GPU-free, requiring only a single forward pass, and can be seamlessly integrated into existing MLLMS.
  arXiv  Detail & Related papers  (2025-09-19T15:30:26Z)
• ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models [68.46716645478661]
  Vision-language models (VLMs) have demonstrated remarkable capabilities in understanding and reasoning about visual content.<n>Current VLMs excel primarily at egocentric spatial reasoning (from the camera's perspective) but fail to generalize to allocentric viewpoints.<n>We introduce ViewSpatial-Bench, the first comprehensive benchmark designed specifically for multi-viewpoint spatial localization recognition evaluation.
  arXiv  Detail & Related papers  (2025-05-27T17:59:26Z)
• SpaRE: Enhancing Spatial Reasoning in Vision-Language Models with Synthetic Data [7.142118464319378]
  Vision-language models (VLMs) work well in tasks ranging from image captioning to visual question answering (VQA)<n>We find that spatial relations are generally rare in widely used VL datasets, with only a few being well represented.<n>We construct a synthetic VQA dataset focused on spatial reasoning generated from hyper-detailed image descriptions.
  arXiv  Detail & Related papers  (2025-04-29T11:18:38Z)
• SpaceR: Reinforcing MLLMs in Video Spatial Reasoning [70.7401015322983]
  Video spatial reasoning poses a significant challenge for existing Multimodal Large Language Models (MLLMs)<n>This limitation stems primarily from 1) the absence of high-quality datasets for this task, and 2) the lack of effective training strategies to develop spatial reasoning capabilities.<n>Motivated by the success of Reinforcement Learning with Verifiable Reward (RLVR) in unlocking spatial reasoning abilities, this aims to improve MLLMs in video spatial reasoning through the RLVR paradigm.
  arXiv  Detail & Related papers  (2025-04-02T15:12:17Z)
• Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models [14.442394137843923]
  We present a detailed analysis that first delineates the core elements of spatial reasoning.<n>We then assesses the performance of these models in both synthetic and real-world images.
  arXiv  Detail & Related papers  (2025-03-25T14:34:06Z)
• TopViewRS: Vision-Language Models as Top-View Spatial Reasoners [38.406430696146714]
  Top-view perspective denotes a typical way in which humans read and reason over different types of maps. We introduce the TopViewRS dataset, consisting of 11,384 multiple-choice questions with either realistic or semantic top-view map as visual input. We then use it to study and evaluate VLMs across 4 perception and reasoning tasks with different levels of complexity.
  arXiv  Detail & Related papers  (2024-06-04T17:55:43Z)
• SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models [68.13636352687257]
  We introduce Spatial Region GPT (SpatialRGPT) to enhance VLMs' spatial perception and reasoning capabilities. During inference, when provided with user-specified region proposals, SpatialRGPT can accurately perceive their relative directions and distances. Our results demonstrate that SpatialRGPT significantly enhances performance in spatial reasoning tasks, both with and without local region prompts.
  arXiv  Detail & Related papers  (2024-06-03T17:59:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.