Beyond Similarity Matching: Structured Reasoning for Open-Vocabulary Referring Segmentation in 3DGS

• URL: http://arxiv.org/abs/2608.16103v1
• Date: Mon, 17 Aug 2026 04:48:02 GMT
• Title: Beyond Similarity Matching: Structured Reasoning for Open-Vocabulary Referring Segmentation in 3DGS
• Abstract summary: We propose QAGaussian, a query-adaptive neural reasoning framework for language-guided Gaussian primitive selection.<n>It learns query-conditioned multi-scale Gaussian slots as differentiable candidates whose receptive fields are shaped by the input expression.<n>It then builds a relation-aware slot graph with language-conditioned edge weighting to propagate target-reference, attribute, part-whole, and contextual evidence.<n>It achieves 47.2 Avg. mIoU and 63.2 Avg. F1, outperforming the strongest 3DGS referring baseline by 2.7 mIoU points and 2.9 F1 points.
• Score: 7.483297048216368
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Open-vocabulary referring segmentation in 3D Gaussian Splatting (3DGS) requires a neural model to select Gaussian primitives according to free-form language expressions. Existing 3DGS-based methods usually rely on global text-region similarity, which is weak for queries involving attributes, reference objects, spatial relations, and fine-grained parts. This often causes target-reference confusion, granularity mismatch, part-whole leakage, and relation violations. We propose QAGaussian, a query-adaptive neural reasoning framework for language-guided Gaussian primitive selection. QAGaussian first learns query-conditioned multi-scale Gaussian slots as differentiable candidates whose receptive fields are shaped by the input expression. It then builds a relation-aware slot graph with language-conditioned edge weighting to propagate target-reference, attribute, part-whole, and contextual evidence. A granularity-adaptive router softly combines region-level, object-level, part-level, attribute-aware, and relation-aware mask branches, followed by relation-constrained refinement for spatial, part-whole, attribute, and geometric consistency. QAGaussian is pretrained only on Mosaic3D-5.6M for Gaussian-text alignment and evaluated on independent benchmarks without target-dataset fine-tuning. It achieves 47.2 Avg. mIoU and 63.2 Avg. F1, outperforming the strongest 3DGS referring baseline by 2.7 mIoU points and 2.9 F1 points. It also improves Part-mIoU from 38.6 to 43.4, Rel-mIoU from 44.4 to 50.8, and reduces target-reference confusion from 10.8 to 7.4. These results demonstrate that query-conditioned slot learning, relation-aware graph reasoning, and adaptive routing provide an effective neural modeling strategy for open-vocabulary referring segmentation in 3DGS. The code is available at https://github.com/zqeslwyz/QAGaussian.

Related papers

• Gaussian-JEPA: Joint-Embedding Predictive Learning for 3D Gaussian Splats [91.55220894541151]
  3D Gaussian Splatting (3DGS) represents 3D content with anisotropic primitives that jointly encode geometry and appearance.<n>We introduce Gaussian-JEPA, which predicts representations of held-out Gaussian token blocks from visible context.
  arXiv  Detail & Related papers  (2026-08-16T09:43:50Z)
• Relation-Centric Open-Vocabulary 3D Gaussian Segmentation [31.791638071734337]
  PairGS is a framework that reframes Gaussian segmentation as modeling pairwise relations between Gaussians.<n>It achieves state-of-the-art results on open-vocabulary 3D Gaussian segmentation benchmarks, while the fast variant is 50x faster than optimization-based instance-feature approaches.
  arXiv  Detail & Related papers  (2026-07-01T16:23:41Z)
• ULF-Loc: Unbiased Landmark Feature for Robust Visual Localization with 3D Gaussian Splatting [14.25287150227726]
  We propose ULF-Loc, an unbiased landmark feature framework that replaces biased feature optimization with geometry-weighted feature fusion.<n>On the Cambridge Landmarks dataset, ULF-Loc reduces the mean median translation error by 17% compared to the state-of-the-art.
  arXiv  Detail & Related papers  (2026-05-06T10:30:50Z)
• NG-GS: NeRF-Guided 3D Gaussian Splatting Segmentation [65.34304674634713]
  We introduce NG-GS, a framework for high-quality object segmentation in 3DGS that explicitly addresses boundary discretization.<n>We show that our method achieves photorealistic state-of-the-art performance, with significant gains in boundary mIoU.
  arXiv  Detail & Related papers  (2026-04-16T07:14:07Z)
• LabelGS: Label-Aware 3D Gaussian Splatting for 3D Scene Segmentation [56.4321049923868]
  3D Gaussian Splatting (3DGS) has emerged as a novel explicit representation for 3D scenes, offering both high-fidelity reconstruction and efficient rendering.<n>We propose Label-aware 3D Gaussian Splatting (LabelGS), a method that augments the Gaussian representation with object label.<n>LabelGS achieves a remarkable 22X speedup in training compared to Feature-3DGS, at a resolution of 1440X1080.
  arXiv  Detail & Related papers  (2025-08-27T09:07:38Z)
• GraphGSOcc: Semantic-Geometric Graph Transformer with Dynamic-Static Decoupling for 3D Gaussian Splatting-based Occupancy Prediction [2.3239379129613535]
  GraphGSOcc is a novel framework that combines semantic and geometric graph Transformer and decouples dynamic-static objects.<n>It achieves state-ofthe-art performance on the SurroundOcc-nuScenes, Occ3D-nuScenes, OpenOcc and KITTI occupancy benchmarks.
  arXiv  Detail & Related papers  (2025-06-13T06:09:57Z)
• FlexGS: Train Once, Deploy Everywhere with Many-in-One Flexible 3D Gaussian Splatting [57.97160965244424]
  3D Gaussian splatting (3DGS) has enabled various applications in 3D scene representation and novel view synthesis.<n>Previous approaches have focused on pruning less important Gaussians, effectively compressing 3DGS.<n>We present an elastic inference method for 3DGS, achieving substantial rendering performance without additional fine-tuning.
  arXiv  Detail & Related papers  (2025-06-04T17:17:57Z)
• CAGS: Open-Vocabulary 3D Scene Understanding with Context-Aware Gaussian Splatting [18.581169318975046]
  3D Gaussian Splatting (3DGS) offers a powerful representation for scene reconstruction, but cross-view granularity inconsistency is a problem.<n>We propose Context-Aware Gaussian Splatting (CAGS), a novel framework that addresses this challenge by incorporating spatial context into 3DGS.<n>CAGS significantly improves 3D instance segmentation and reduces fragmentation errors on datasets like LERF-OVS and ScanNet.
  arXiv  Detail & Related papers  (2025-04-16T09:20:03Z)
• ShapeSplat: A Large-scale Dataset of Gaussian Splats and Their Self-Supervised Pretraining [112.40071212468843]
  3D Gaussian Splatting (3DGS) has become the de facto method of 3D representation in many vision tasks.<n>We build a large-scale dataset of 3DGS using the commonly used ShapeNet, ModelNet and averse.<n>We introduce Gaussian-MAE, which highlights the unique benefits of representation learning from Gaussian parameters.
  arXiv  Detail & Related papers  (2024-08-20T14:49:14Z)
• GOI: Find 3D Gaussians of Interest with an Optimizable Open-vocabulary Semantic-space Hyperplane [53.388937705785025]
  3D open-vocabulary scene understanding is crucial for advancing augmented reality and robotic applications. We introduce GOI, a framework that integrates semantic features from 2D vision-language foundation models into 3D Gaussian Splatting (3DGS) Our method treats the feature selection process as a hyperplane division within the feature space, retaining only features that are highly relevant to the query.
  arXiv  Detail & Related papers  (2024-05-27T18:57:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.