MonoVoc: Decoupling Geometry and Semantics for Lightweight Monocular Open-Vocabulary 3D Gaussians

• URL: http://arxiv.org/abs/2607.28300v1
• Date: Thu, 30 Jul 2026 14:41:02 GMT
• Title: MonoVoc: Decoupling Geometry and Semantics for Lightweight Monocular Open-Vocabulary 3D Gaussians
• Abstract summary: Open vocabulary 3D scene understanding is essential for next-generation interactive systems, empowering users to intuitively query and navigate reconstructed environments using natural language.<n>We present a novel, training-free pipeline that fundamentally reimagines this paradigm by explicitly decoupling 3D geometric reconstruction from semantic integration.
• Score: 10.451279636801301
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Open vocabulary 3D scene understanding is essential for next-generation interactive systems, empowering users to intuitively query and navigate reconstructed environments using natural language. However, current 3D Gaussian frameworks are often bottlenecked by restrictive multiview capture requirements, costly scene-specific optimization, and the massive memory overhead of storing dense language features. We present a novel, training-free pipeline that fundamentally reimagines this paradigm by explicitly decoupling 3D geometric reconstruction from semantic integration. Given a standard monocular video sequence as input, our method efficiently outputs a compact, highly interpretable, and fully searchable object-level semantic Gaussian map. Rather than entangling heavy language embeddings within the mapping loop, we extract geometry independently and ground semantics through a lightweight, modular post-processing framework. Extensive evaluations on the Replica dataset demonstrate that this decoupled architecture preserves strong rendering fidelity and competitive segmentation accuracy. Crucially, by replacing dense per-Gaussian storage with modular, object-level semantic embeddings, our approach delivers an order-of-magnitude reduction in memory usage compared to SOTA baselines. This provides a highly efficient, scalable, and practical solution for open-vocabulary 3D retrieval and question answering directly from everyday monocular video.

Related papers

• QueryGaussian: Scalable and Training-Free Open-Vocabulary 3D Instance Retrieval [19.833205230045074]
  QueryGaussian is a training-free framework for expeditious and scalable open-vocabulary 3D instance retrieval.<n>We leverage pre-trained 2D vision models to interpret user prompts and lift segmentation masks into 3D.<n>We also introduce a temporal fusion module with multi-stage adaptive density clustering to mitigate projection ambiguity.
  arXiv  Detail & Related papers  (2026-06-18T02:57:35Z)
• LangFlash: Feed-forward 3D Language Gaussian Splatting from Sparse Unposed Images [30.52329450141629]
  LangFlash is a feed-forward framework for 3D Language Gaussian Splatting.<n>LangFlash directly predicts the geometry and semantics in a single forward pass.<n>We propose a sparse semantic encoding scheme that combines a global semantic dictionary with locally varying per-primitive weights.
  arXiv  Detail & Related papers  (2026-05-22T06:59:00Z)
• OpenGaFF: Open-Vocabulary Gaussian Feature Field with Codebook Attention [80.51557267896938]
  We present OpenGaFF, a novel framework for open-vocabulary 3D scene understanding built upon 3D Gaussian Splatting.<n>At the core of our method is a Gaussian Feature Field that models semantics as a continuous function of Gaussian geometry and appearance.<n>To further enforce object-level semantic consistency, we introduce a structured codebook that serves as a set of shared semantic primitives.
  arXiv  Detail & Related papers  (2026-05-07T12:10:07Z)
• Point Cloud as a Foreign Language for Multi-modal Large Language Model [22.63554774942836]
  SAGE is the first end-to-end 3D MLLM that directly processes raw point clouds without relying on a pre-trained 3D encoder.<n>Our approach introduces a lightweight 3D tokenizer that combines geometric sampling and neighbourhood aggregation with vector quantization to convert point clouds into discrete tokens.
  arXiv  Detail & Related papers  (2026-03-10T04:22:40Z)
• FLEG: Feed-Forward Language Embedded Gaussian Splatting from Any Views [52.02871618456553]
  FLEG is a feed-forward network that reconstructs language-embedded 3D Gaussians from any views.<n>We propose a 3D-annotation-free training framework for 2D-to-3D lifting from arbitrary uncalibrated and unposed multi-view images.
  arXiv  Detail & Related papers  (2025-12-19T13:04:13Z)
• C3G: Learning Compact 3D Representations with 2K Gaussians [55.04010158339562]
  Recent approaches use per-pixel 3D Gaussian Splatting for reconstruction, followed by a 2D-to-3D feature lifting stage for scene understanding.<n>We propose C3G, a novel feed-forward framework that estimates compact 3D Gaussians only at essential spatial locations.
  arXiv  Detail & Related papers  (2025-12-03T17:59:05Z)
• SegSplat: Feed-forward Gaussian Splatting and Open-Set Semantic Segmentation [114.57192386025373]
  SegSplat is a novel framework designed to bridge the gap between rapid, feed-forward 3D reconstruction and rich, open-vocabulary semantic understanding.<n>This work represents a significant step towards practical, on-the-fly generation of semantically aware 3D environments.
  arXiv  Detail & Related papers  (2025-11-23T10:26:38Z)
• OpenGS-Fusion: Open-Vocabulary Dense Mapping with Hybrid 3D Gaussian Splatting for Refined Object-Level Understanding [17.524454394142477]
  We present OpenGS-Fusion, an innovative open-vocabulary dense mapping framework that improves semantic modeling and refines object-level understanding.<n>We also introduce a novel multimodal language-guided approach named MLLM-Assisted Adaptive Thresholding, which refines the segmentation of 3D objects by adaptively adjusting similarity thresholds.<n>Our method outperforms existing methods in 3D object understanding and scene reconstruction quality, as well as showcasing its effectiveness in language-guided scene interaction.
  arXiv  Detail & Related papers  (2025-08-02T02:22:36Z)
• Training-Free Hierarchical Scene Understanding for Gaussian Splatting with Superpoint Graphs [16.153129392697885]
  We introduce a training-free framework that constructs a superpoint graph directly from Gaussian primitives.<n>The superpoint graph partitions the scene into spatially compact and semantically coherent regions, forming view-consistent 3D entities.<n>Our method achieves state-of-the-art open-vocabulary segmentation performance, with semantic field reconstruction completed over $30times$ faster.
  arXiv  Detail & Related papers  (2025-04-17T17:56:07Z)
• Large Spatial Model: End-to-end Unposed Images to Semantic 3D [79.94479633598102]
  Large Spatial Model (LSM) processes unposed RGB images directly into semantic radiance fields. LSM simultaneously estimates geometry, appearance, and semantics in a single feed-forward operation. It can generate versatile label maps by interacting with language at novel viewpoints.
  arXiv  Detail & Related papers  (2024-10-24T17:54:42Z)
• Convolutional Occupancy Networks [88.48287716452002]
  We propose Convolutional Occupancy Networks, a more flexible implicit representation for detailed reconstruction of objects and 3D scenes. By combining convolutional encoders with implicit occupancy decoders, our model incorporates inductive biases, enabling structured reasoning in 3D space. We empirically find that our method enables the fine-grained implicit 3D reconstruction of single objects, scales to large indoor scenes, and generalizes well from synthetic to real data.
  arXiv  Detail & Related papers  (2020-03-10T10:17:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.