Not All Relations Rotate Alike: Transformation-Aware Decoupling for Viewpoint-Robust 3D Scene Graph Generation

• URL: http://arxiv.org/abs/2606.27412v1
• Date: Thu, 25 Jun 2026 14:44:40 GMT
• Title: Not All Relations Rotate Alike: Transformation-Aware Decoupling for Viewpoint-Robust 3D Scene Graph Generation
• Abstract summary: 3D Scene Graph Generation (3DSGG) represents 3D scenes as structured object-relation-object graphs.<n>Current 3DSGG models often fail to produce relation predictions that follow the expected transformation behavior under viewpoint shifts.<n>We propose Transformation-Aware Decoupling (TAD), a viewpoint-robust 3DSGG framework that decouples relation reasoning according to predicate transformation behavior.
• Score: 11.312501987929224
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: 3D Scene Graph Generation (3DSGG) represents 3D scenes as structured object-relation-object graphs, providing a compact relational abstraction for spatial understanding. In embodied intelligence settings, the same 3D scene may be observed by agents from viewpoints that differ by yaw rotations. However, current 3DSGG models often fail to produce relation predictions that follow the expected transformation behavior under such viewpoint shifts. This behavior reveals an empirical mismatch related to predicate-level transformation heterogeneity: directional predicates such as left, front, right, and behind should transform with the observation frame, whereas most contact, support, and semantic predicates such as standing on and attached to should remain stable. To reduce this mismatch, we propose Transformation-Aware Decoupling (TAD), a viewpoint-robust 3DSGG framework that decouples relation reasoning according to predicate transformation behavior and is supported by viewpoint-stable object representations. TAD decomposes relation reasoning into two parts: one learns cues that should stay stable across viewpoints, while the other learns directional cues that should change with the observation frame. The two parts are merged for standard multi-label predicate prediction. Transformation-specific descriptors and group-aware auxiliary supervision encourage the two branches to capture complementary relation cues. Extensive experiments on 3DSSG show that TAD achieves state-of-the-art robustness under yaw viewpoint changes without training-time rotation augmentation, while maintaining competitive performance under the standard benchmark. The project page is available at https://tad-predicate.github.io/.

Related papers

• Beyond Isolated Objects: Relationship-aware Open Vocabulary Scene Understanding via 3D Scene Graph Analysis [59.527737790821305]
  We propose a relationship-aware framework that uses 3D scene graphs to enhance open-vocabulary 3D understanding.<n>Our method constructs relational scene graphs from multi-view observations by leveraging vision-language reasoning to infer object relationships.<n>Experiments on ScanNetV2, ScanNet200, ScanNet$++$, and Replica demonstrate strong performance and generalization ability.
  arXiv  Detail & Related papers  (2026-07-06T17:29:49Z)
• RelWitness: Open-Vocabulary 3D Scene Graph Generation with Visual-Geometric Relation Witnesses [1.3468350096927395]
  We propose a framework for open-vocabulary 3D scene graph generation from posed RGB-D sequences under incomplete relation supervision.<n>The key concept is a relation witness: a concrete visual-geometric cue that makes a relation observable in the captured scene.<n> RelWitness constructs relation witness records from RGB views, depth maps, reconstructed 3D geometry, role-sensitive text, object-prior null views, and multi-view consistency.
  arXiv  Detail & Related papers  (2026-05-20T07:18:56Z)
• 3D Segmentation Using Viewpoint-Dependent Spatial Relationships [55.198821645924234]
  We introduce a viewpoint-aware 3D segmentation dataset containing 220k benchmark samples.<n>In this dataset, target objects can only be identified through observer-centric spatial relations.<n>We introduce a viewpoint representation that encodes camera poses and conditions the model on the observation viewpoint.
  arXiv  Detail & Related papers  (2026-05-15T07:58:44Z)
• VIZOR: Viewpoint-Invariant Zero-Shot Scene Graph Generation for 3D Scene Reasoning [1.9190955990713918]
  We propose Viewpoint-Invariant Zero-shot scene graph generation for 3D scene Reasoning (VIZOR)<n>VIZOR is a training-free, end-to-end framework that constructs dense, viewpoint-invariant 3D scene graphs directly from raw 3D scenes.<n>It infers open-vocabulary relationships that describe spatial and proximity relationships among scene objects without requiring annotated training data.
  arXiv  Detail & Related papers  (2026-01-31T10:11:27Z)
• Edge-Centric Relational Reasoning for 3D Scene Graph Prediction [74.19580969696898]
  3D scene graph prediction aims to abstract complex 3D environments into structured graphs consisting of objects and their pairwise relationships.<n>Existing approaches typically adopt object-centric graph neural networks, where relation edge features are iteratively updated by aggregating messages from connected object nodes.<n>We propose a Link-guided Edge-centric relational reasoning framework with Object-aware fusion.
  arXiv  Detail & Related papers  (2025-11-19T09:53:56Z)
• IGGT: Instance-Grounded Geometry Transformer for Semantic 3D Reconstruction [82.53307702809606]
  Humans naturally perceive the geometric structure and semantic content of a 3D world as intertwined dimensions.<n>We propose InstanceGrounded Geometry Transformer (IGGT) to unify the knowledge for both spatial reconstruction and instance-level contextual understanding.
  arXiv  Detail & Related papers  (2025-10-26T14:57:44Z)
• Bootstraping Clustering of Gaussians for View-consistent 3D Scene Understanding [59.51535163599723]
  FreeGS is an unsupervised semantic-embedded 3DGS framework that achieves view-consistent 3D scene understanding without the need for 2D labels.<n>FreeGS performs comparably to state-of-the-art methods while avoiding the complex data preprocessing workload.
  arXiv  Detail & Related papers  (2024-11-29T08:52:32Z)
• S^2Former-OR: Single-Stage Bi-Modal Transformer for Scene Graph Generation in OR [50.435592120607815]
  Scene graph generation (SGG) of surgical procedures is crucial in enhancing holistically cognitive intelligence in the operating room (OR) Previous works have primarily relied on multi-stage learning, where the generated semantic scene graphs depend on intermediate processes with pose estimation and object detection. In this study, we introduce a novel single-stage bi-modal transformer framework for SGG in the OR, termed S2Former-OR.
  arXiv  Detail & Related papers  (2024-02-22T11:40:49Z)
• Unsupervised Object Representation Learning using Translation and Rotation Group Equivariant VAE [5.254093731341154]
  TARGET-VAE is a translation and rotation group-equivariant variational autoencoder framework. We show that TARGET-VAE learns disentangled representations without supervision that significantly improve upon, and avoid the pathologies of, previous methods. It is able to perform remarkably accurate unsupervised pose and location inference.
  arXiv  Detail & Related papers  (2022-10-24T02:08:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.