RelWitness: Open-Vocabulary 3D Scene Graph Generation with Visual-Geometric Relation Witnesses

• URL: http://arxiv.org/abs/2605.20823v2
• Date: Thu, 21 May 2026 01:40:41 GMT
• Title: RelWitness: Open-Vocabulary 3D Scene Graph Generation with Visual-Geometric Relation Witnesses
• Abstract summary: We propose a framework for open-vocabulary 3D scene graph generation from posed RGB-D sequences under incomplete relation supervision.<n>The key concept is a relation witness: a concrete visual-geometric cue that makes a relation observable in the captured scene.<n> RelWitness constructs relation witness records from RGB views, depth maps, reconstructed 3D geometry, role-sensitive text, object-prior null views, and multi-view consistency.
• Score: 1.3468350096927395
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Open-vocabulary 3D scene graph generation seeks to describe object instances and their relations with flexible natural-language predicates. The central difficulty is not only vocabulary expansion, but supervision reliability: relation annotations in 3D scene graph datasets are selective, and many valid object-pair relations are unannotated. We propose RelWitness, a framework for open-vocabulary 3D scene graph generation from posed RGB-D sequences under incomplete relation supervision. The key concept is a relation witness: a concrete visual-geometric cue that makes a relation observable in the captured scene. Support relations require contact and vertical ordering; containment requires enclosure; proximity requires metric closeness; orientation requires facing direction; and stable relations should persist across views where both objects are visible. RelWitness constructs relation witness records from RGB views, depth maps, reconstructed 3D geometry, role-sensitive text, object-prior null views, and multi-view consistency. A visual-geometric witness verifier assigns unannotated relation candidates to verified missing positives, reliable negatives, or uncertain unlabeled cases. A witness-guided positive-unlabeled objective then learns from incomplete annotations without turning every missing label into a negative. We further introduce witness-consistent decoding and an RGB-D missing-relation audit protocol. Simulated manuscript-planning experiments on 3DSSG/3RScan and ScanNet-derived open-vocabulary splits show the intended behavior: improved unseen-relation recognition, higher witness precision, lower hallucination, and reduced redundant relation phrases. All numerical results are planning values and must be replaced by reproduced measurements before submission

Related papers

• Beyond Isolated Objects: Relationship-aware Open Vocabulary Scene Understanding via 3D Scene Graph Analysis [59.527737790821305]
  We propose a relationship-aware framework that uses 3D scene graphs to enhance open-vocabulary 3D understanding.<n>Our method constructs relational scene graphs from multi-view observations by leveraging vision-language reasoning to infer object relationships.<n>Experiments on ScanNetV2, ScanNet200, ScanNet$++$, and Replica demonstrate strong performance and generalization ability.
  arXiv  Detail & Related papers  (2026-07-06T17:29:49Z)
• Not All Relations Rotate Alike: Transformation-Aware Decoupling for Viewpoint-Robust 3D Scene Graph Generation [11.312501987929224]
  3D Scene Graph Generation (3DSGG) represents 3D scenes as structured object-relation-object graphs.<n>Current 3DSGG models often fail to produce relation predictions that follow the expected transformation behavior under viewpoint shifts.<n>We propose Transformation-Aware Decoupling (TAD), a viewpoint-robust 3DSGG framework that decouples relation reasoning according to predicate transformation behavior.
  arXiv  Detail & Related papers  (2026-06-25T14:44:40Z)
• ReLIC-SGG: Relation Lattice Completion for Open-Vocabulary Scene Graph Generation [5.184088650563149]
  We propose textbfReLIC-SGG, a relation-incompleteness-aware framework for open-vocabulary scene graph generation.<n>ReLIC-SGG builds a semantic relation lattice to model similarity, entailment, and contradiction among open-vocabulary predicates.<n> Experiments show that ReLIC-SGG improves rare and unseen predicate recognition and better recovers missing relations.
  arXiv  Detail & Related papers  (2026-04-24T13:36:41Z)
• CAGE-SGG: Counterfactual Active Graph Evidence for Open-Vocabulary Scene Graph Generation [16.971992237358638]
  Open-vocabulary scene graph generation (SGG) aims to describe visual scenes with flexible and fine-grained relation phrases.<n>We propose an evidence-rounded open-vocabulary SGG framework based on counterfactual relation verification.
  arXiv  Detail & Related papers  (2026-04-24T06:34:45Z)
• VIZOR: Viewpoint-Invariant Zero-Shot Scene Graph Generation for 3D Scene Reasoning [1.9190955990713918]
  We propose Viewpoint-Invariant Zero-shot scene graph generation for 3D scene Reasoning (VIZOR)<n>VIZOR is a training-free, end-to-end framework that constructs dense, viewpoint-invariant 3D scene graphs directly from raw 3D scenes.<n>It infers open-vocabulary relationships that describe spatial and proximity relationships among scene objects without requiring annotated training data.
  arXiv  Detail & Related papers  (2026-01-31T10:11:27Z)
• Edge-Centric Relational Reasoning for 3D Scene Graph Prediction [74.19580969696898]
  3D scene graph prediction aims to abstract complex 3D environments into structured graphs consisting of objects and their pairwise relationships.<n>Existing approaches typically adopt object-centric graph neural networks, where relation edge features are iteratively updated by aggregating messages from connected object nodes.<n>We propose a Link-guided Edge-centric relational reasoning framework with Object-aware fusion.
  arXiv  Detail & Related papers  (2025-11-19T09:53:56Z)
• Do It Yourself: Learning Semantic Correspondence from Pseudo-Labels [81.22943696917304]
  We propose improving semantic correspondence estimation through 3D-aware pseudo-labeling.<n>We establish a new state-of-the-art on SPair-71k, achieving an absolute gain of over 4% and of over 7% compared to methods with similar supervision requirements.
  arXiv  Detail & Related papers  (2025-06-05T17:54:33Z)
• Generalized Visual Relation Detection with Diffusion Models [94.62313788626128]
  Visual relation detection (VRD) aims to identify relationships (or interactions) between object pairs in an image.<n>We propose to model visual relations as continuous embeddings, and design diffusion models to achieve generalized VRD in a conditional generative manner.<n>Our Diff-VRD is able to generate visual relations beyond the pre-defined category labels of datasets.
  arXiv  Detail & Related papers  (2025-04-16T14:03:24Z)
• Open-Vocabulary Octree-Graph for 3D Scene Understanding [54.11828083068082]
  Octree-Graph is a novel scene representation for open-vocabulary 3D scene understanding. An adaptive-octree structure is developed that stores semantics and depicts the occupancy of an object adjustably according to its shape.
  arXiv  Detail & Related papers  (2024-11-25T10:14:10Z)
• Relation Rectification in Diffusion Model [64.84686527988809]
  We introduce a novel task termed Relation Rectification, aiming to refine the model to accurately represent a given relationship it initially fails to generate. We propose an innovative solution utilizing a Heterogeneous Graph Convolutional Network (HGCN) The lightweight HGCN adjusts the text embeddings generated by the text encoder, ensuring the accurate reflection of the textual relation in the embedding space.
  arXiv  Detail & Related papers  (2024-03-29T15:54:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.