DeWorldSG: Depth-Aware 3D Semantic Scene Graph Generation via World-Model Priors

• URL: http://arxiv.org/abs/2607.00889v1
• Date: Wed, 01 Jul 2026 12:55:09 GMT
• Title: DeWorldSG: Depth-Aware 3D Semantic Scene Graph Generation via World-Model Priors
• Abstract summary: Existing methods often struggle to construct reliable 3D scene graphs due to unstable 3D object representations and missing relations caused by frame-wise inference.<n>DeWorldSG addresses these issues by instance estimating geometric-level 3D Gaussian distributions through depth-guided filtering.<n> Experiments on the 3DSSG and ReplicaSSSG datasets demonstrate state-of-the-art (SoTA) performance in both object and prediction.
• Score: 22.5705525097147
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: We present DeWorldSG, a novel framework that generates spatio-temporally robust 3D Semantic Scene Graphs from RGB-D sequences. Existing methods often struggle to construct reliable 3D scene graphs due to unstable 3D object representations and missing relations caused by frame-wise inference. DeWorldSG addresses these issues by estimating instance-level geometric 3D Gaussian distributions through depth-guided filtering and representing each object as a probabilistic 3D node rather than a single projected point. To mitigate relational sparsity from frame-wise inference, our framework further aggregates spatiotemporal evidence across object pairs and refines relations using contextual priors derived from a world model (V-JEPA 2). Experiments on the 3DSSG and ReplicaSSG datasets demonstrate state-of-the-art (SoTA) performance in both object and predicate prediction, while producing temporally consistent scene structures. In particular, our method improves triplet recall by 77.4% and predicate recall by 23.2% over prior SoTA approaches, making it suitable for robotic manipulation and AR applications. Our code and models are open-sourced.

Related papers

• NoPA: Non-Parametric Online 3D Scene Graph Generation [57.15076373014265]
  NoPA represents each object as a separate non-parametric distribution.<n>We leverage maximum mean discrepancy on kernel density estimates to enable robust merging of object candidates.<n>NoPA substantially outperforms current methods without sacrificing real-time inference speed.
  arXiv  Detail & Related papers  (2026-07-01T07:18:07Z)
• Graph-GSReg: Leveraging 3D Scene Graphs for Gaussian Splatting Registration [21.225937272344968]
  Merging multiple 3D Gaussian Splatting (3DGS) scenes into a single unified representation is essential for large-scale 3D mapping and map management.<n>Previous methods attempt direct correspondence between Gaussian primitives and require training on large 3DGS.<n>Our method constructs a 3D scene graph from a 3DGS and its rendered images, textitreformulating 3DGS registration as a graph registration problem
  arXiv  Detail & Related papers  (2026-06-29T04:58:44Z)
• 3D-Fixer: Coarse-to-Fine In-place Completion for 3D Scenes from a Single Image [26.04490259188974]
  We introduce 3D-Fixer, a novel in-place completion paradigm for compositional 3D scene generation.<n>Unlike prior works that require explicit pose alignment, 3D-Fixer uses fragmented geometry as a spatial anchor to preserve layout fidelity.<n>We present ARSG-110K, the largest scene-level dataset to date, comprising over 110K diverse scenes and 3M annotated images with high-fidelity 3D ground truth.
  arXiv  Detail & Related papers  (2026-04-06T04:11:09Z)
• Action-Geometry Prediction with 3D Geometric Prior for Bimanual Manipulation [53.09168514034483]
  Bimanual manipulation requires policies that can reason about 3D geometry, anticipate how it evolves under action, and generate smooth, coordinated motions.<n>We propose a framework that builds bimanual manipulation directly on a pre-trained 3D geometric foundation model.<n>Our policy fuses geometry-aware latents, 2D semantic features, and proprioception into a unified state representation, and uses diffusion model to jointly predict a future action chunk and a future 3D latent that decodes into a dense pointmap.
  arXiv  Detail & Related papers  (2026-02-27T08:54:20Z)
• ReferSplat: Referring Segmentation in 3D Gaussian Splatting [60.73702075842278]
  Referring 3D Gaussian Splatting (R3DGS)<n>Task aims to segment target objects in a 3D Gaussian scene based on natural language descriptions.<n>To address these challenges, we propose ReferSplat, a framework that explicitly models 3D Gaussian points with natural language expressions.
  arXiv  Detail & Related papers  (2025-08-11T17:59:30Z)
• Bootstraping Clustering of Gaussians for View-consistent 3D Scene Understanding [59.51535163599723]
  FreeGS is an unsupervised semantic-embedded 3DGS framework that achieves view-consistent 3D scene understanding without the need for 2D labels.<n>FreeGS performs comparably to state-of-the-art methods while avoiding the complex data preprocessing workload.
  arXiv  Detail & Related papers  (2024-11-29T08:52:32Z)
• GSD: View-Guided Gaussian Splatting Diffusion for 3D Reconstruction [52.04103235260539]
  We present a diffusion model approach based on Gaussian Splatting representation for 3D object reconstruction from a single view. The model learns to generate 3D objects represented by sets of GS ellipsoids. The final reconstructed objects explicitly come with high-quality 3D structure and texture, and can be efficiently rendered in arbitrary views.
  arXiv  Detail & Related papers  (2024-07-05T03:43:08Z)
• Incremental 3D Semantic Scene Graph Prediction from RGB Sequences [86.77318031029404]
  We propose a real-time framework that incrementally builds a consistent 3D semantic scene graph of a scene given an RGB image sequence. Our method consists of a novel incremental entity estimation pipeline and a scene graph prediction network. The proposed network estimates 3D semantic scene graphs with iterative message passing using multi-view and geometric features extracted from the scene entities.
  arXiv  Detail & Related papers  (2023-05-04T11:32:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.