Related papers: GST-Bench: Can VLMs Develop Global Spatial Awareness from Video?

GST-Bench: Can VLMs Develop Global Spatial Awareness from Video?

URL: http://arxiv.org/abs/2608.05747v1
Date: Thu, 06 Aug 2026 08:33:13 GMT
Title: GST-Bench: Can VLMs Develop Global Spatial Awareness from Video?
Abstract summary: GST-Bench is a benchmark for global spatial intelligence in video understanding.<n>It requires models to perform accurate spatial inference from novel viewpoints unseen in the input video.<n>The strongest zero-shot model attains only 42.68, far below the human score of 79.08.
Score: 28.125428186179573
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: Spatial intelligence is fundamental to embodied agents, yet existing benchmarks focus on local spatial perception from single or few viewpoints, overlooking global spatial awareness over continuous, long-horizon visual streams. To address this limitation, we introduce the Global-Spatial-Temporal Benchmark (GST-Bench), a VQA benchmark for global spatial intelligence in video understanding, comprising human-verified questions derived from 6,790 minutes of synthetically generated video. It requires models to perform accurate spatial inference from novel viewpoints unseen in the input video and to map egocentric observations onto global top-down images. A comprehensive evaluation of 22 state-of-the-art VLMs exposes a striking gap between models and humans: the strongest zero-shot model attains only 42.68, far below the human score of 79.08. To probe the cause of this gap, we construct GST-Bench-Local and find that models, despite strong local spatial understanding under the same task formulation, still fail to consolidate long-horizon observations into a globally consistent scene representation. We further provide GST-Train, a dataset for global spatial reasoning, as a complementary resource to facilitate future research on this challenge.

Related papers

Towards Spatial Supersensing in the Wild [35.413360698005285]
We introduce $textbfVSI-Super-Wild, a benchmark for evaluating spatial supersensing over long temporal horizons.<n>Results show that models consistently fail at tasks that require coherent world-state tracking over time.<n>We characterize how performance degrades with world-state and temporal horizon, and diagnose four failure modes: spatial collapse, semantic shortcuts, complexity update, and instance confusion.
arXiv Detail & Related papers (2026-07-15T10:24:51Z)
OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMs [49.82435858489898]
OVOS-Bench is a fully-annotated benchmark for streaming human-centric spatial intelligence.<n>It consists of 1,680 questions on 348 source videos.<n>Each question carries a query and an evidence interval, and at evaluation, the model sees only the prefix preceding the query.
arXiv Detail & Related papers (2026-06-02T16:51:32Z)
Are VLMs Lost Between Sky and Space? LinkS$^2$Bench for UAV-Satellite Dynamic Cross-View Spatial Intelligence [107.79496500634691]
LinkS$2$Bench links 1,022 minutes of dynamic UAV footage with high-resolution satellite imagery covering over 200 km$2$.<n>Through an LMM-assisted pipeline and rigorous human annotation, we constructed 17.9k high-quality question-answer pairs.<n> Evaluations reveal a substantial gap compared to human baselines, identifying accurate cross-view dynamic alignment as the critical bottleneck.
arXiv Detail & Related papers (2026-04-02T13:22:57Z)
TagaVLM: Topology-Aware Global Action Reasoning for Vision-Language Navigation [70.23578202012048]
Vision-Language Navigation (VLN) presents a unique challenge for Large Vision-Language Models (VLMs) due to their inherent architectural mismatch.<n>We propose TagaVLM (Topology-Aware Global Action reasoning), an end-to-end framework that explicitly injects topological structures into the VLM backbone.<n>To enhance topological node information, an Interleaved Navigation Prompt strengthens node-level visual-text alignment.<n>With the embedded topological graph, the model is capable of global action reasoning, allowing for robust path correction.
arXiv Detail & Related papers (2026-03-03T13:28:07Z)
OmniGround: A Comprehensive Spatio-Temporal Grounding Benchmark for Real-World Complex Scenarios [39.58602686069029]
We introduce OmniGround, a comprehensive benchmark with 3,475 videos spanning 81 categories and complex real-world queries.<n>We also introduce DeepSTG, a systematic evaluation framework quantifying dataset quality across four complementary dimensions.<n>Experiments demonstrate PG-TAF achieves 25.6% and 35.6% improvements in m_tIoU and m_vIoU with consistent gains across four benchmarks.
arXiv Detail & Related papers (2025-11-21T04:23:04Z)
Cambrian-S: Towards Spatial Supersensing in Video [78.46305169769884]
We frame spatial supersensing as four stages beyond linguistic-only understanding: semantic perception, streaming event cognition, implicit 3D spatial cognition, and predictive world modeling.<n>To drive progress in spatial supersensing, we present-SUPER, a two-part benchmark: VSR (long-horizon visual spatial recall) and VSC (continual visual spatial counting)<n>We then test data scaling limits by curating-590K and training Cambrian-S, achieving +30% absolute improvement on arbitrarily general capabilities.<n>We propose predictive sensing as a path forward, presenting a proof-of-concept in which a self-supervised
arXiv Detail & Related papers (2025-11-06T18:55:17Z)
Where on Earth? A Vision-Language Benchmark for Probing Model Geolocation Skills Across Scales [61.03549470159347]
Vision-language models (VLMs) have advanced rapidly, yet their capacity for image-grounded geolocation in open-world conditions has not been comprehensively evaluated.<n>We present EarthWhere, a comprehensive benchmark for VLM image geolocation that evaluates visual recognition, step-by-step reasoning, and evidence use.
arXiv Detail & Related papers (2025-10-13T01:12:21Z)
GTR-Bench: Evaluating Geo-Temporal Reasoning in Vision-Language Models [12.634203010453282]
Geo-Temporal Reasoning benchmark (GTR-Bench) is a novel challenge for geographic temporal reasoning of moving targets in a large-scale camera network.<n> Evaluations of more than 10 popular Visual-Language Models (VLMs) on GTR-Bench demonstrate that even the best proprietary model, Gemini-2.5-Pro, significantly lags behind human performance (78.61%) on geo-temporal reasoning.<n>GTR-Bench offers valuable insights and opens up new opportunities for research and applications in spatial-temporal intelligence.
arXiv Detail & Related papers (2025-10-09T05:09:27Z)
Jigsaw-Puzzles: From Seeing to Understanding to Reasoning in Vision-Language Models [12.945689517235264]
We introduce Jigsaw-Puzzles, a novel benchmark consisting of 1,100 carefully curated real-world images with high spatial complexity.<n>Based on this dataset, we design five tasks to rigorously evaluate vision-language models' spatial perception, structural understanding, and reasoning capabilities.<n>The results show that even the strongest model, Gemini-2.5-Pro, achieves only 77.14% overall accuracy and performs particularly poorly on the Order Generation task.
arXiv Detail & Related papers (2025-05-27T05:17:41Z)
SpaceVLLM: Endowing Multimodal Large Language Model with Spatio-Temporal Video Grounding Capability [58.46310813774538]
Large language models (LMLMs) have made remarkable progress in either temporal or spatial localization.<n>However they struggle to perform-temporal video grounding.<n>This limitation stems from two major challenges.<n>We introduce SpaceLM, a MLLMVL endowed with temporal-temporal video grounding.
arXiv Detail & Related papers (2025-03-18T07:40:36Z)
Watching You: Global-guided Reciprocal Learning for Video-based Person Re-identification [82.6971648465279]
We propose a novel Global-guided Reciprocal Learning framework for video-based person Re-ID. Our approach can achieve better performance than other state-of-the-art approaches.
arXiv Detail & Related papers (2021-03-07T12:27:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.