Spatially Invariant Unsupervised 3D Object Segmentation with Graph Neural Networks

• URL: http://arxiv.org/abs/2106.05607v2
• Date: Fri, 11 Jun 2021 12:07:16 GMT
• Title: Spatially Invariant Unsupervised 3D Object Segmentation with Graph Neural Networks
• Authors: Tianyu Wang, Miaomiao Liu, Kee Siong Ng
• Abstract summary: We propose a framework, SPAIR3D, to model a point cloud as a spatial mixture model. We jointly learn the multiple-object representation and segmentation in 3D via Variational Autoencoders (VAE) Experimental results demonstrate that SPAIR3D is capable of detecting and segmenting variable number of objects without appearance information.
• Score: 23.729853358582506
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In this paper, we tackle the problem of unsupervised 3D object segmentation from a point cloud without RGB information. In particular, we propose a framework, SPAIR3D, to model a point cloud as a spatial mixture model and jointly learn the multiple-object representation and segmentation in 3D via Variational Autoencoders (VAE). Inspired by SPAIR, we adopt an object-specification scheme that describes each object's location relative to its local voxel grid cell rather than the point cloud as a whole. To model the spatial mixture model on point clouds, we derive the Chamfer Likelihood, which fits naturally into the variational training pipeline. We further design a new spatially invariant graph neural network to generate a varying number of 3D points as a decoder within our VAE. Experimental results demonstrate that SPAIR3D is capable of detecting and segmenting variable number of objects without appearance information across diverse scenes.

Related papers

• Explicit3D: Graph Network with Spatial Inference for Single Image 3D Object Detection [35.85544715234846]
  We propose a dynamic sparse graph pipeline named Explicit3D based on object geometry and semantics features. Our experimental results on the SUN RGB-D dataset demonstrate that our Explicit3D achieves better performance balance than the-state-of-the-art.
  arXiv  Detail & Related papers  (2023-02-13T16:19:54Z)
• ONeRF: Unsupervised 3D Object Segmentation from Multiple Views [59.445957699136564]
  ONeRF is a method that automatically segments and reconstructs object instances in 3D from multi-view RGB images without any additional manual annotations. The segmented 3D objects are represented using separate Neural Radiance Fields (NeRFs) which allow for various 3D scene editing and novel view rendering.
  arXiv  Detail & Related papers  (2022-11-22T06:19:37Z)
• Geometry-Contrastive Transformer for Generalized 3D Pose Transfer [95.56457218144983]
  The intuition of this work is to perceive the geometric inconsistency between the given meshes with the powerful self-attention mechanism. We propose a novel geometry-contrastive Transformer that has an efficient 3D structured perceiving ability to the global geometric inconsistencies. We present a latent isometric regularization module together with a novel semi-synthesized dataset for the cross-dataset 3D pose transfer task.
  arXiv  Detail & Related papers  (2021-12-14T13:14:24Z)
• Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based Perception [122.53774221136193]
  State-of-the-art methods for driving-scene LiDAR-based perception often project the point clouds to 2D space and then process them via 2D convolution. A natural remedy is to utilize the 3D voxelization and 3D convolution network. We propose a new framework for the outdoor LiDAR segmentation, where cylindrical partition and asymmetrical 3D convolution networks are designed to explore the 3D geometric pattern.
  arXiv  Detail & Related papers  (2021-09-12T06:25:11Z)
• Learning Geometry-Disentangled Representation for Complementary Understanding of 3D Object Point Cloud [50.56461318879761]
  We propose Geometry-Disentangled Attention Network (GDANet) for 3D image processing. GDANet disentangles point clouds into contour and flat part of 3D objects, respectively denoted by sharp and gentle variation components. Experiments on 3D object classification and segmentation benchmarks demonstrate that GDANet achieves the state-of-the-arts with fewer parameters.
  arXiv  Detail & Related papers  (2020-12-20T13:35:00Z)
• ODFNet: Using orientation distribution functions to characterize 3D point clouds [0.0]
  We leverage on point orientation distributions around a point in order to obtain an expressive local neighborhood representation for point clouds. New ODFNet model achieves state-of-the-art accuracy for object classification on ModelNet40 and ScanObjectNN datasets.
  arXiv  Detail & Related papers  (2020-12-08T19:54:20Z)
• Exploring Deep 3D Spatial Encodings for Large-Scale 3D Scene Understanding [19.134536179555102]
  We propose an alternative approach to overcome the limitations of CNN based approaches by encoding the spatial features of raw 3D point clouds into undirected graph models. The proposed method achieves on par state-of-the-art accuracy with improved training time and model stability thus indicating strong potential for further research.
  arXiv  Detail & Related papers  (2020-11-29T12:56:19Z)
• Spherical Interpolated Convolutional Network with Distance-Feature Density for 3D Semantic Segmentation of Point Clouds [24.85151376535356]
  Spherical interpolated convolution operator is proposed to replace the traditional grid-shaped 3D convolution operator. The proposed method achieves good performance on the ScanNet dataset and Paris-Lille-3D dataset.
  arXiv  Detail & Related papers  (2020-11-27T15:35:12Z)
• Reinforced Axial Refinement Network for Monocular 3D Object Detection [160.34246529816085]
  Monocular 3D object detection aims to extract the 3D position and properties of objects from a 2D input image. Conventional approaches sample 3D bounding boxes from the space and infer the relationship between the target object and each of them, however, the probability of effective samples is relatively small in the 3D space. We propose to start with an initial prediction and refine it gradually towards the ground truth, with only one 3d parameter changed in each step. This requires designing a policy which gets a reward after several steps, and thus we adopt reinforcement learning to optimize it.
  arXiv  Detail & Related papers  (2020-08-31T17:10:48Z)
• Stereo RGB and Deeper LIDAR Based Network for 3D Object Detection [40.34710686994996]
  3D object detection has become an emerging task in autonomous driving scenarios. Previous works process 3D point clouds using either projection-based or voxel-based models. We propose the Stereo RGB and Deeper LIDAR framework which can utilize semantic and spatial information simultaneously.
  arXiv  Detail & Related papers  (2020-06-09T11:19:24Z)
• Cylindrical Convolutional Networks for Joint Object Detection and Viewpoint Estimation [76.21696417873311]
  We introduce a learnable module, cylindrical convolutional networks (CCNs), that exploit cylindrical representation of a convolutional kernel defined in the 3D space. CCNs extract a view-specific feature through a view-specific convolutional kernel to predict object category scores at each viewpoint. Our experiments demonstrate the effectiveness of the cylindrical convolutional networks on joint object detection and viewpoint estimation.
  arXiv  Detail & Related papers  (2020-03-25T10:24:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.