Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networks

• URL: http://arxiv.org/abs/2108.07478v1
• Date: Tue, 17 Aug 2021 07:25:14 GMT
• Title: Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networks
• Authors: Zhihao Liang, Zhihao Li, Songcen Xu, Mingkui Tan and Kui Jia
• Abstract summary: We propose an end-to-end solution of Semantic Superpoint Tree Network (SSTNet) for proposing object instances from scene points. Key in SSTNet is an intermediate, semantic superpoint tree (SST), which is constructed based on the learned semantic features of superpoints. SSTNet ranks top on the ScanNet (V2) leaderboard, with 2% higher of mAP than the second best method.
• Score: 64.27814530457042
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Instance segmentation in 3D scenes is fundamental in many applications of scene understanding. It is yet challenging due to the compound factors of data irregularity and uncertainty in the numbers of instances. State-of-the-art methods largely rely on a general pipeline that first learns point-wise features discriminative at semantic and instance levels, followed by a separate step of point grouping for proposing object instances. While promising, they have the shortcomings that (1) the second step is not supervised by the main objective of instance segmentation, and (2) their point-wise feature learning and grouping are less effective to deal with data irregularities, possibly resulting in fragmented segmentations. To address these issues, we propose in this work an end-to-end solution of Semantic Superpoint Tree Network (SSTNet) for proposing object instances from scene points. Key in SSTNet is an intermediate, semantic superpoint tree (SST), which is constructed based on the learned semantic features of superpoints, and which will be traversed and split at intermediate tree nodes for proposals of object instances. We also design in SSTNet a refinement module, termed CliqueNet, to prune superpoints that may be wrongly grouped into instance proposals. Experiments on the benchmarks of ScanNet and S3DIS show the efficacy of our proposed method. At the time of submission, SSTNet ranks top on the ScanNet (V2) leaderboard, with 2% higher of mAP than the second best method. The source code in PyTorch is available at https://github.com/Gorilla-Lab-SCUT/SSTNet.

Related papers

• LESS: Label-Efficient and Single-Stage Referring 3D Segmentation [55.06002976797879]
  Referring 3D is a visual-language task that segments all points of the specified object from a 3D point cloud described by a sentence of query. We propose a novel Referring 3D pipeline, Label-Efficient and Single-Stage, dubbed LESS, which is only under the supervision of efficient binary mask. We achieve state-of-the-art performance on ScanRefer dataset by surpassing the previous methods about 3.7% mIoU using only binary labels.
  arXiv  Detail & Related papers  (2024-10-17T07:47:41Z)
• SegPoint: Segment Any Point Cloud via Large Language Model [62.69797122055389]
  We propose a model, called SegPoint, to produce point-wise segmentation masks across a diverse range of tasks. SegPoint is the first model to address varied segmentation tasks within a single framework.
  arXiv  Detail & Related papers  (2024-07-18T17:58:03Z)
• SPGroup3D: Superpoint Grouping Network for Indoor 3D Object Detection [23.208654655032955]
  Current 3D object detection methods for indoor scenes mainly follow the voting-and-grouping strategy to generate proposals. We propose a novel superpoint grouping network for indoor anchor-free one-stage 3D object detection. Experimental results demonstrate our method achieves state-of-the-art performance on ScanNet V2, SUN RGB-D, and S3DIS datasets.
  arXiv  Detail & Related papers  (2023-12-21T08:08:02Z)
• Superpoint Transformer for 3D Scene Instance Segmentation [7.07321040534471]
  This paper proposes a novel end-to-end 3D instance segmentation method based on Superpoint Transformer, named as SPFormer. It groups potential features from point clouds into superpoints, and directly predicts instances through query vectors. It exceeds compared state-of-the-art methods by 4.3% on ScanNetv2 hidden test set in terms of mAP and keeps fast inference speed (247ms per frame) simultaneously.
  arXiv  Detail & Related papers  (2022-11-28T20:52:53Z)
• Divide and Conquer: 3D Point Cloud Instance Segmentation With Point-Wise Binarization [16.662238192665615]
  We propose a novel divide-and-conquer strategy named PBNet for segmenting point clouds. Our binary clustering divides offset instance points into two categories: high and low density points. PBNet ranks first on the ScanNetV2 official benchmark challenge, achieving the highest mAP.
  arXiv  Detail & Related papers  (2022-07-22T17:19:00Z)
• PointInst3D: Segmenting 3D Instances by Points [136.7261709896713]
  We propose a fully-convolutional 3D point cloud instance segmentation method that works in a per-point prediction fashion. We find the key to its success is assigning a suitable target to each sampled point. Our approach achieves promising results on both ScanNet and S3DIS benchmarks.
  arXiv  Detail & Related papers  (2022-04-25T02:41:46Z)
• GSIP: Green Semantic Segmentation of Large-Scale Indoor Point Clouds [64.86292006892093]
  GSIP (Green of Indoor Point clouds) is an efficient solution to semantic segmentation of large-scale indoor scene point clouds. GSIP has two novel components: 1) a room-style data pre-processing method that selects a proper subset of points for further processing, and 2) a new feature extractor which is extended from PointHop. Experiments show that GSIP outperforms PointNet in segmentation performance for the S3DIS dataset.
  arXiv  Detail & Related papers  (2021-09-24T09:26:53Z)
• Hierarchical Aggregation for 3D Instance Segmentation [41.20244892803604]
  We propose a clustering-based framework named HAIS, which makes full use of spatial relation of points and point sets. It ranks 1st on the ScanNet v2 benchmark, achieving the highest 69.9% AP50 and surpassing previous state-of-the-art (SOTA) methods by a large margin.
  arXiv  Detail & Related papers  (2021-08-05T03:34:34Z)
• FPCC: Fast Point Cloud Clustering for Instance Segmentation [4.007351600492542]
  There has been little research on 3D point cloud instance segmentation of bin-picking scenes. We propose a network (FPCC-Net) that infers feature centers of each instance and then clusters the remaining points. It is shown that FPCC-Net improves average precision (AP) by about 40% and can process about 60,000 points in about 0.8 [s]
  arXiv  Detail & Related papers  (2020-12-29T05:58:35Z)
• PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation [111.7241018610573]
  We present PointGroup, a new end-to-end bottom-up architecture for instance segmentation. We design a two-branch network to extract point features and predict semantic labels and offsets, for shifting each point towards its respective instance centroid. A clustering component is followed to utilize both the original and offset-shifted point coordinate sets, taking advantage of their complementary strength. We conduct extensive experiments on two challenging datasets, ScanNet v2 and S3DIS, on which our method achieves the highest performance, 63.6% and 64.0%, compared to 54.9% and 54.4% achieved by former best
  arXiv  Detail & Related papers  (2020-04-03T16:26:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.