ViewMind3D: Modular View-Aware Inference for Training-Free 3D-QA

• URL: http://arxiv.org/abs/2607.28442v1
• Date: Thu, 30 Jul 2026 16:14:30 GMT
• Title: ViewMind3D: Modular View-Aware Inference for Training-Free 3D-QA
• Abstract summary: textbfViewMind3D is a fully training-free and modular framework for 3D spatial reasoning over multi-view observations of a scene.<n>We show that ViewMind3D achieves competitive performance compared to prior training-free and fine-tuned 3D-LLMs.
• Score: 26.947027118604368
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in large language models (LLMs) and vision-language models (VLMs) have enabled new possibilities for 3D question answering (3D-QA), a key capability for embodied AI and robotic perception. However, most existing methods rely on 3D-specific training or fine-tuning with costly annotations, limiting their scalability and real-world applicability. We present \textbf{ViewMind3D}, a fully training-free and modular framework for 3D spatial reasoning over multi-view observations of a scene without requiring complete 3D reconstruction. The framework decomposes the 3D-QA task into four interpretable components: (1) question-driven multi-view selection, (2) guided visual grounding with language-conditioned object cues, (3) spatial context encoding via a bird's-eye-view (BEV) viewpoint indicator, and (4) structured answer generation through role-based reasoning. This design enables structured, robust, and interpretable reasoning without requiring model tuning. Experimental results on ScanQA and SQA3D show that ViewMind3D achieves competitive performance compared to prior training-free and fine-tuned 3D-LLMs. In particular, our method improves performance on spatially grounded question types, such as ``What'' questions in SQA3D, while maintaining strong overall accuracy (50.8\%) and achieving 73.4 CIDEr on ScanQA. These results demonstrate that effective 3D reasoning can be achieved through modular orchestration of general-purpose LLMs and VLMs for robotic perception in real-world environments.

Related papers

• Do 3D Large Language Models Really Understand 3D Spatial Relationships? [80.64317885117704]
  3D Large-Language Models claim to understand 3D worlds, especially spatial relationships among objects.<n>We find that fine-tuning a language model on text-only question-answer pairs can perform comparably or even surpass these methods on the SQA3D benchmark without using any 3D input.<n>We introduce Real-3DQA, a more rigorous evaluation benchmark that filters out easy-to-guess questions and introduces a structured taxonomy to assess various aspects of 3D reasoning.
  arXiv  Detail & Related papers  (2026-03-06T16:04:34Z)
• Abstract 3D Perception for Spatial Intelligence in Vision-Language Models [100.13033631690114]
  Vision-language models (VLMs) struggle with 3D-related tasks such as spatial cognition and physical understanding.<n>We introduce SandboxVLM, a framework that leverages abstract bounding boxes to encode geometric structure and physical kinematics for VLM.<n>Our approach consistently improves spatial intelligence, achieving an 8.3% gain on SAT Real compared with baseline methods.
  arXiv  Detail & Related papers  (2025-11-14T04:16:09Z)
• Aligning Text, Images, and 3D Structure Token-by-Token [8.521599463802637]
  We investigate the potential of autoregressive models for structured 3D scenes.<n>We propose a unified LLM framework that aligns language, images, and 3D scenes.<n>We show our model's effectiveness on real-world 3D object recognition tasks.
  arXiv  Detail & Related papers  (2025-06-09T17:59:37Z)
• 3D Question Answering via only 2D Vision-Language Models [87.41421075243103]
  Large vision-language models (LVLMs) have advanced numerous fields.<n>We explore how to harness their potential to address 3D scene understanding tasks, using 3D question answering (3D-QA) as a representative example.<n>Specifically, we sample 2D views from a 3D point cloud and feed them into 2D models to answer a given question.<n>We propose cdViews, a novel approach to automatically selecting critical and diverse Views for 3D-QA.
  arXiv  Detail & Related papers  (2025-05-28T09:04:39Z)
• VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction [86.82819259860186]
  We introduce VLM-3R, a unified framework for Vision-Language Models (VLMs) that incorporates 3D Reconstructive instruction tuning.<n>VLM-3R processes monocular video frames by employing a geometry encoder to derive implicit 3D tokens that represent spatial understanding.
  arXiv  Detail & Related papers  (2025-05-26T17:56:30Z)
• Unveiling the Mist over 3D Vision-Language Understanding: Object-centric Evaluation with Chain-of-Analysis [65.42684641776931]
  3D vision-language (3D-VL) benchmarks fall short in evaluating 3D-VL models.<n>We propose Beacon3D, a benchmark for 3D-VL grounding and QA tasks.
  arXiv  Detail & Related papers  (2025-03-28T13:32:29Z)
• 3D-Aware Visual Question Answering about Parts, Poses and Occlusions [20.83938624671415]
  We introduce the task of 3D-aware VQA, which focuses on challenging questions that require a compositional reasoning over the 3D structure of visual scenes. We propose PO3D-VQA, a 3D-aware VQA model that marries two powerful ideas: probabilistic neural symbolic program execution for reasoning and deep neural networks with 3D generative representations of objects for robust visual recognition. Our experimental results show our model PO3D-VQA outperforms existing methods significantly, but we still observe a significant performance gap compared to 2D VQA benchmarks.
  arXiv  Detail & Related papers  (2023-10-27T06:15:30Z)
• Multi-CLIP: Contrastive Vision-Language Pre-training for Question Answering tasks in 3D Scenes [68.61199623705096]
  Training models to apply common-sense linguistic knowledge and visual concepts from 2D images to 3D scene understanding is a promising direction that researchers have only recently started to explore. We propose a novel 3D pre-training Vision-Language method, namely Multi-CLIP, that enables a model to learn language-grounded and transferable 3D scene point cloud representations.
  arXiv  Detail & Related papers  (2023-06-04T11:08:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.