PinpointQA: A Dataset and Benchmark for Small Object-Centric Spatial Understanding in Indoor Videos

• URL: http://arxiv.org/abs/2604.08991v1
• Date: Fri, 10 Apr 2026 05:53:47 GMT
• Title: PinpointQA: A Dataset and Benchmark for Small Object-Centric Spatial Understanding in Indoor Videos
• Abstract summary: PinpointQA is the first dataset and benchmark for small object-centric spatial understanding in indoor videos.<n>It comprises 1,024 scenes and 10,094 QA pairs organized into four progressively challenging tasks.<n>Experiments reveal a consistent capability gap along the progressive chain, with SSP remaining particularly difficult.
• Score: 23.951493516327016
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Small object-centric spatial understanding in indoor videos remains a significant challenge for multimodal large language models (MLLMs), despite its practical value for object search and assistive applications. Although existing benchmarks have advanced video spatial intelligence, embodied reasoning, and diagnostic perception, no existing benchmark directly evaluates whether a model can localize a target object in video and express its position with sufficient precision for downstream use. In this work, we introduce PinpointQA, the first dataset and benchmark for small object-centric spatial understanding in indoor videos. Built from ScanNet++ and ScanNet200, PinpointQA comprises 1,024 scenes and 10,094 QA pairs organized into four progressively challenging tasks: Target Presence Verification (TPV), Nearest Reference Identification (NRI), Fine-Grained Spatial Description (FSD), and Structured Spatial Prediction (SSP). The dataset is built from intermediate spatial representations, with QA pairs generated automatically and further refined through quality control. Experiments on representative MLLMs reveal a consistent capability gap along the progressive chain, with SSP remaining particularly difficult. Supervised fine-tuning on PinpointQA yields substantial gains, especially on the harder tasks, demonstrating that PinpointQA serves as both a diagnostic benchmark and an effective training dataset. The dataset and project page are available at https://rainchowz.github.io/PinpointQA.

Related papers

• SPARROW: Learning Spatial Precision and Temporal Referential Consistency in Pixel-Grounded Video MLLMs [39.73103140338364]
  We introduce SPARROW, a pixel-grounded video MLLM that unifies spatial accuracy and temporal stability through two key components.<n> SPARROW is supported by a curated referential video dataset of 30,646 videos and 45,231 Q&A pairs.<n>It delivers consistent gains across six benchmarks, improving up to +8.9 J&F on RVOS, +5 mIoU on visual grounding, and +5.4 CLAIR on GCG.
  arXiv  Detail & Related papers  (2026-03-12T18:59:57Z)
• Reasoning with Pixel-level Precision: QVLM Architecture and SQuID Dataset for Quantitative Geospatial Analytics [0.2291770711277359]
  Current Vision-Language Models (VLMs) fail at quantitative spatial reasoning because their architectures destroy pixel-level information required for counting and measurements.<n>We present two contributions to address this fundamental limitation. First, we introduce SQuID, a benchmark of 2,000 satellite image Question-Answer pairs with both numerical range and categorical answers.<n>Second, we propose QVLM, a code-generation architecture that maintains pixel precision by decoupling language understanding from visual analysis.
  arXiv  Detail & Related papers  (2026-01-19T21:14:34Z)
• From Indoor to Open World: Revealing the Spatial Reasoning Gap in MLLMs [65.04549036809557]
  We introduce a benchmark built from pedestrian-perspective videos captured with synchronized stereo cameras, LiDAR, and IMU/GPS sensors.<n>This dataset provides metrically precise 3D information, enabling the automatic generation of spatial reasoning questions.<n> Evaluations reveal that the performance gains observed in structured indoor benchmarks vanish in open-world settings.
  arXiv  Detail & Related papers  (2025-12-22T18:58:12Z)
• OST-Bench: Evaluating the Capabilities of MLLMs in Online Spatio-temporal Scene Understanding [50.72259772580637]
  We introduce OST-Bench, a benchmark designed to evaluate Online Spatio-Temporal understanding from the perspective of an agent.<n>Built on an efficient data collection pipeline, OST-Bench consists of 1.4k scenes and 10k question-answer pairs collected from ScanNet, Matterport3D, and ARKitScenes.<n>We find that both complex-based spatial reasoning demands and long-term memory retrieval requirements significantly drop model performance along two separate axes.
  arXiv  Detail & Related papers  (2025-07-10T17:56:07Z)
• Box-QAymo: Box-Referring VQA Dataset for Autonomous Driving [27.39309272688527]
  Interpretable communication is essential for safe and trustworthy autonomous driving.<n>Current vision-language models (VLMs) often operate under idealized assumptions and struggle to capture user intent in real-world scenarios.<n>Box-QAymo is a box-referring dataset and benchmark designed to evaluate robustness and finetune VLMs on spatial and temporal reasoning over user-specified objects.
  arXiv  Detail & Related papers  (2025-07-01T07:40:16Z)
• ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models [68.46716645478661]
  Vision-language models (VLMs) have demonstrated remarkable capabilities in understanding and reasoning about visual content.<n>Current VLMs excel primarily at egocentric spatial reasoning (from the camera's perspective) but fail to generalize to allocentric viewpoints.<n>We introduce ViewSpatial-Bench, the first comprehensive benchmark designed specifically for multi-viewpoint spatial localization recognition evaluation.
  arXiv  Detail & Related papers  (2025-05-27T17:59:26Z)
• SpatialScore: Towards Unified Evaluation for Multimodal Spatial Understanding [64.15606979785355]
  Multimodal large language models (MLLMs) have achieved impressive success in question-answering tasks, yet their capabilities for spatial understanding are less explored.<n>This work investigates a critical question: do existing MLLMs possess 3D spatial perception and understanding abilities?
  arXiv  Detail & Related papers  (2025-05-22T17:59:03Z)
• GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Tasks [84.86699025256705]
  We present GEOBench-VLM, a benchmark specifically designed to evaluate Vision-Language Models (VLMs) on geospatial tasks.<n>Our benchmark features over 10,000 manually verified instructions and spanning diverse visual conditions, object types, and scales.<n>We evaluate several state-of-the-art VLMs to assess performance on geospatial-specific challenges.
  arXiv  Detail & Related papers  (2024-11-28T18:59:56Z)
• MMScan: A Multi-Modal 3D Scene Dataset with Hierarchical Grounded Language Annotations [55.022519020409405]
  This paper builds the first largest ever multi-modal 3D scene dataset and benchmark with hierarchical grounded language annotations, MMScan.<n>The resulting multi-modal 3D dataset encompasses 1.4M meta-annotated captions on 109k objects and 7.7k regions as well as over 3.04M diverse samples for 3D visual grounding and question-answering benchmarks.
  arXiv  Detail & Related papers  (2024-06-13T17:59:30Z)
• Knowing Earlier what Right Means to You: A Comprehensive VQA Dataset for Grounding Relative Directions via Multi-Task Learning [16.538887534958555]
  We introduce GRiD-A-3D, a novel diagnostic visual question-answering dataset based on abstract objects. Our dataset allows for a fine-grained analysis of end-to-end VQA models' capabilities to ground relative directions. We demonstrate that within a few epochs, the subtasks required to reason over relative directions are learned in the order in which relative directions are intuitively processed.
  arXiv  Detail & Related papers  (2022-07-06T12:31:49Z)
• Dynamic Refinement Network for Oriented and Densely Packed Object Detection [75.29088991850958]
  We present a dynamic refinement network that consists of two novel components, i.e., a feature selection module (FSM) and a dynamic refinement head (DRH) Our FSM enables neurons to adjust receptive fields in accordance with the shapes and orientations of target objects, whereas the DRH empowers our model to refine the prediction dynamically in an object-aware manner. We perform quantitative evaluations on several publicly available benchmarks including DOTA, HRSC2016, SKU110K, and our own SKU110K-R dataset.
  arXiv  Detail & Related papers  (2020-05-20T11:35:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.