SG-Reg: Generalizable and Efficient Scene Graph Registration

• URL: http://arxiv.org/abs/2504.14440v1
• Date: Sun, 20 Apr 2025 01:22:40 GMT
• Title: SG-Reg: Generalizable and Efficient Scene Graph Registration
• Authors: Chuhao Liu, Zhijian Qiao, Jieqi Shi, Ke Wang, Peize Liu, Shaojie Shen,
• Abstract summary: We design a scene graph network to encode multiple modalities of semantic nodes.<n>In the back-end, we employ a robust pose estimator to decide transformation according to the correspondences.<n>Our method achieves a slightly higher registration recall while requiring only 52 KB of communication bandwidth for each query frame.
• Score: 23.3853919684438
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper addresses the challenges of registering two rigid semantic scene graphs, an essential capability when an autonomous agent needs to register its map against a remote agent, or against a prior map. The hand-crafted descriptors in classical semantic-aided registration, or the ground-truth annotation reliance in learning-based scene graph registration, impede their application in practical real-world environments. To address the challenges, we design a scene graph network to encode multiple modalities of semantic nodes: open-set semantic feature, local topology with spatial awareness, and shape feature. These modalities are fused to create compact semantic node features. The matching layers then search for correspondences in a coarse-to-fine manner. In the back-end, we employ a robust pose estimator to decide transformation according to the correspondences. We manage to maintain a sparse and hierarchical scene representation. Our approach demands fewer GPU resources and fewer communication bandwidth in multi-agent tasks. Moreover, we design a new data generation approach using vision foundation models and a semantic mapping module to reconstruct semantic scene graphs. It differs significantly from previous works, which rely on ground-truth semantic annotations to generate data. We validate our method in a two-agent SLAM benchmark. It significantly outperforms the hand-crafted baseline in terms of registration success rate. Compared to visual loop closure networks, our method achieves a slightly higher registration recall while requiring only 52 KB of communication bandwidth for each query frame. Code available at: \href{http://github.com/HKUST-Aerial-Robotics/SG-Reg}{http://github.com/HKUST-Aerial-Robotics/SG-Reg}.

Related papers

• A-SCoRe: Attention-based Scene Coordinate Regression for wide-ranging scenarios [1.2093553114715083]
  A-ScoRe is an Attention-based model which leverage attention on descriptor map level to produce meaningful and high-semantic 2D descriptors. Results show our methods achieve comparable performance with State-of-the-art methods on multiple benchmark while being light-weighted and much more flexible.
  arXiv  Detail & Related papers  (2025-03-18T07:39:50Z)
• Open-Vocabulary Octree-Graph for 3D Scene Understanding [54.11828083068082]
  Octree-Graph is a novel scene representation for open-vocabulary 3D scene understanding. An adaptive-octree structure is developed that stores semantics and depicts the occupancy of an object adjustably according to its shape.
  arXiv  Detail & Related papers  (2024-11-25T10:14:10Z)
• ZeroReg: Zero-Shot Point Cloud Registration with Foundation Models [77.84408427496025]
  State-of-the-art 3D point cloud registration methods rely on labeled 3D datasets for training.<n>We introduce ZeroReg, a zero-shot registration approach that utilizes 2D foundation models to predict 3D correspondences.
  arXiv  Detail & Related papers  (2023-12-05T11:33:16Z)
• Dynamic Graph Message Passing Networks for Visual Recognition [112.49513303433606]
  Modelling long-range dependencies is critical for scene understanding tasks in computer vision. A fully-connected graph is beneficial for such modelling, but its computational overhead is prohibitive. We propose a dynamic graph message passing network, that significantly reduces the computational complexity.
  arXiv  Detail & Related papers  (2022-09-20T14:41:37Z)
• SARNet: Semantic Augmented Registration of Large-Scale Urban Point Clouds [19.41446935340719]
  We propose SARNet, a novel semantic augmented registration network for urban point clouds. Our approach fully exploits semantic features as assistance to improve registration accuracy. We evaluate the proposed SARNet extensively by using real-world data from large regions of urban scenes.
  arXiv  Detail & Related papers  (2022-06-27T08:49:11Z)
• SIGMA: Semantic-complete Graph Matching for Domain Adaptive Object Detection [26.0630601028093]
  Domain Adaptive Object Detection (DAOD) leverages a labeled domain to learn an object detector generalizing to a novel domain free of annotations. Recent advances align class-conditional distributions by narrowing down cross-domain prototypes (class centers) We propose a novel SemantIc-complete Graph MAtching framework for hallucinationD, which completes mismatched semantics and reformulates the adaptation with graph matching.
  arXiv  Detail & Related papers  (2022-03-12T10:14:17Z)
• Group-Wise Semantic Mining for Weakly Supervised Semantic Segmentation [49.90178055521207]
  This work addresses weakly supervised semantic segmentation (WSSS), with the goal of bridging the gap between image-level annotations and pixel-level segmentation. We formulate WSSS as a novel group-wise learning task that explicitly models semantic dependencies in a group of images to estimate more reliable pseudo ground-truths. In particular, we devise a graph neural network (GNN) for group-wise semantic mining, wherein input images are represented as graph nodes.
  arXiv  Detail & Related papers  (2020-12-09T12:40:13Z)
• Towards Efficient Scene Understanding via Squeeze Reasoning [71.1139549949694]
  We propose a novel framework called Squeeze Reasoning. Instead of propagating information on the spatial map, we first learn to squeeze the input feature into a channel-wise global vector. We show that our approach can be modularized as an end-to-end trained block and can be easily plugged into existing networks.
  arXiv  Detail & Related papers  (2020-11-06T12:17:01Z)
• Representative Graph Neural Network [113.67254049938629]
  We present a Representative Graph layer to dynamically sample a few representative features. Instead of propagating the messages from all positions, our RepGraph layer computes the response of one node merely with a few representative nodes.
  arXiv  Detail & Related papers  (2020-08-12T09:46:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.