SpatialQ: Understanding 3D Gaussian Splatting Scene Quality via Visual-based MLLM

• URL: http://arxiv.org/abs/2607.26595v1
• Date: Wed, 29 Jul 2026 08:14:40 GMT
• Title: SpatialQ: Understanding 3D Gaussian Splatting Scene Quality via Visual-based MLLM
• Abstract summary: A multimodal quality assessment framework is developed for 3DGS scene understanding.<n>A 3D-aware quality representation learning framework is introduced by augmenting a VGGT-based encoder with a dedicated quality head.<n>A grounded multimodal reasoning mechanism is constructed by jointly feeding original images, depth maps, point cloud renderings, and camera parameters into a Qwen-based MLLM.
• Score: 33.48299658365364
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 3D Gaussian Splatting (3DGS) has emerged as an effective representation for novel view synthesis and 3D scene reconstruction, creating an increasing demand for reliable quality assessment. Unlike conventional image quality assessment (IQA), the quality of a 3DGS scene depends not only on the perceptual fidelity of rendered views, but also on scene-level factors such as spatial structure and cross-view consistency. Existing IQA methods are limited by their reliance on 2D perceptual cues, whereas general multimodal large language models (MLLMs) are not designed for stable quality regression and may produce unreliable judgments. To address these limitations, a multimodal quality assessment framework is developed for 3DGS scene understanding. First, a 3D-aware quality representation learning framework is introduced by augmenting a VGGT-based encoder with a dedicated quality head. Multi-view images are encoded into view-specific features and aggregated to capture cross-view consistency, while geometric cues are incorporated through joint modeling of depth and point-cloud-related structural information, enabling the learning of structure-aware quality representations beyond appearance-driven features. Second, a grounded multimodal reasoning mechanism is constructed by jointly feeding original images, depth maps, point cloud renderings, and camera parameters into a Qwen-based MLLM.

Related papers

• Evaluating Foundation Models' 3D Understanding Through Multi-View Correspondence Analysis [38.10984626023432]
  We introduce a novel benchmark for in-context 3D scene understanding that requires no finetuning and directly probes the quality of dense visual features.<n>We benchmark 8 state-of-the-art foundation models and show DINO-based encoders remain competitive across large viewpoint shifts.
  arXiv  Detail & Related papers  (2025-12-12T14:03:16Z)
• IGGT: Instance-Grounded Geometry Transformer for Semantic 3D Reconstruction [82.53307702809606]
  Humans naturally perceive the geometric structure and semantic content of a 3D world as intertwined dimensions.<n>We propose InstanceGrounded Geometry Transformer (IGGT) to unify the knowledge for both spatial reconstruction and instance-level contextual understanding.
  arXiv  Detail & Related papers  (2025-10-26T14:57:44Z)
• Enhancing Novel View Synthesis from extremely sparse views with SfM-free 3D Gaussian Splatting Framework [14.927184256861807]
  We propose a novel SfM-free 3DGS-based method that jointly estimates camera poses and reconstructs 3D scenes from extremely sparse-view inputs.<n>Our method significantly outperforms other state-of-the-art 3DGS-based approaches, achieving a remarkable 2.75dB improvement in PSNR under extremely sparse-view conditions.
  arXiv  Detail & Related papers  (2025-08-21T11:25:24Z)
• Hi3DEval: Advancing 3D Generation Evaluation with Hierarchical Validity [78.7107376451476]
  Hi3DEval is a hierarchical evaluation framework tailored for 3D generative content.<n>We extend texture evaluation beyond aesthetic appearance by explicitly assessing material realism.<n>We propose a 3D-aware automated scoring system based on hybrid 3D representations.
  arXiv  Detail & Related papers  (2025-08-07T17:50:13Z)
• VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction [86.82819259860186]
  We introduce VLM-3R, a unified framework for Vision-Language Models (VLMs) that incorporates 3D Reconstructive instruction tuning.<n>VLM-3R processes monocular video frames by employing a geometry encoder to derive implicit 3D tokens that represent spatial understanding.
  arXiv  Detail & Related papers  (2025-05-26T17:56:30Z)
• M3-AGIQA: Multimodal, Multi-Round, Multi-Aspect AI-Generated Image Quality Assessment [65.3860007085689]
  M3-AGIQA is a comprehensive framework that enables more human-aligned, holistic evaluation of AI-generated images.<n>By aligning model outputs more closely with human judgment, M3-AGIQA delivers robust and interpretable quality scores.
  arXiv  Detail & Related papers  (2025-02-21T03:05:45Z)
• HybridMQA: Exploring Geometry-Texture Interactions for Colored Mesh Quality Assessment [7.526258700061012]
  Mesh quality assessment (MQA) models play a critical role in the design, optimization, and evaluation of mesh operation systems.<n>We introduce HybridMQA, a hybrid full-reference colored MQA framework that integrates model-based and projection-based approaches.<n>Our method employs graph learning to extract detailed 3D representations, which are then projected to 2D using a novel feature rendering process.
  arXiv  Detail & Related papers  (2024-12-02T21:35:33Z)
• No-Reference Point Cloud Quality Assessment via Graph Convolutional Network [89.12589881881082]
  Three-dimensional (3D) point cloud, as an emerging visual media format, is increasingly favored by consumers. Point clouds inevitably suffer from quality degradation and information loss through multimedia communication systems. We propose a novel no-reference PCQA method by using a graph convolutional network (GCN) to characterize the mutual dependencies of multi-view 2D projected image contents.
  arXiv  Detail & Related papers  (2024-11-12T11:39:05Z)
• Q-Ground: Image Quality Grounding with Large Multi-modality Models [61.72022069880346]
  We introduce Q-Ground, the first framework aimed at tackling fine-scale visual quality grounding. Q-Ground combines large multi-modality models with detailed visual quality analysis. Central to our contribution is the introduction of the QGround-100K dataset.
  arXiv  Detail & Related papers  (2024-07-24T06:42:46Z)
• Multi-View Consistent Generative Adversarial Networks for 3D-aware Image Synthesis [48.33860286920389]
  3D-aware image synthesis aims to generate images of objects from multiple views by learning a 3D representation. Existing approaches lack geometry constraints, hence usually fail to generate multi-view consistent images. We propose Multi-View Consistent Generative Adrial Networks (MVCGAN) for high-quality 3D-aware image synthesis with geometry constraints.
  arXiv  Detail & Related papers  (2022-04-13T11:23:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.