Hierarchical Aggregation for 3D Instance Segmentation

• URL: http://arxiv.org/abs/2108.02350v1
• Date: Thu, 5 Aug 2021 03:34:34 GMT
• Title: Hierarchical Aggregation for 3D Instance Segmentation
• Authors: Shaoyu Chen, Jiemin Fang, Qian Zhang, Wenyu Liu, Xinggang Wang
• Abstract summary: We propose a clustering-based framework named HAIS, which makes full use of spatial relation of points and point sets. It ranks 1st on the ScanNet v2 benchmark, achieving the highest 69.9% AP50 and surpassing previous state-of-the-art (SOTA) methods by a large margin.
• Score: 41.20244892803604
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Instance segmentation on point clouds is a fundamental task in 3D scene perception. In this work, we propose a concise clustering-based framework named HAIS, which makes full use of spatial relation of points and point sets. Considering clustering-based methods may result in over-segmentation or under-segmentation, we introduce the hierarchical aggregation to progressively generate instance proposals, i.e., point aggregation for preliminarily clustering points to sets and set aggregation for generating complete instances from sets. Once the complete 3D instances are obtained, a sub-network of intra-instance prediction is adopted for noisy points filtering and mask quality scoring. HAIS is fast (only 410ms per frame) and does not require non-maximum suppression. It ranks 1st on the ScanNet v2 benchmark, achieving the highest 69.9% AP50 and surpassing previous state-of-the-art (SOTA) methods by a large margin. Besides, the SOTA results on the S3DIS dataset validate the good generalization ability. Code will be available at https://github.com/hustvl/HAIS.

Related papers

• ISBNet: a 3D Point Cloud Instance Segmentation Network with Instance-aware Sampling and Box-aware Dynamic Convolution [14.88505076974645]
  ISBNet is a novel method that represents instances as kernels and decodes instance masks via dynamic convolution. We set new state-of-the-art results on ScanNetV2 (55.9), S3DIS (60.8), S3LS3D (49.2) in terms of AP and retains fast inference time (237ms per scene on ScanNetV2.
  arXiv  Detail & Related papers  (2023-03-01T06:06:28Z)
• Divide and Conquer: 3D Point Cloud Instance Segmentation With Point-Wise Binarization [16.662238192665615]
  We propose a novel divide-and-conquer strategy named PBNet for segmenting point clouds. Our binary clustering divides offset instance points into two categories: high and low density points. PBNet ranks first on the ScanNetV2 official benchmark challenge, achieving the highest mAP.
  arXiv  Detail & Related papers  (2022-07-22T17:19:00Z)
• PointInst3D: Segmenting 3D Instances by Points [136.7261709896713]
  We propose a fully-convolutional 3D point cloud instance segmentation method that works in a per-point prediction fashion. We find the key to its success is assigning a suitable target to each sampled point. Our approach achieves promising results on both ScanNet and S3DIS benchmarks.
  arXiv  Detail & Related papers  (2022-04-25T02:41:46Z)
• MaskGroup: Hierarchical Point Grouping and Masking for 3D Instance Segmentation [36.28586460186891]
  This paper studies the 3D instance segmentation problem, which has a variety of real-world applications such as robotics and augmented reality. We propose a novel framework to group and refine the 3D instances. Our approach achieves a 66.4% mAP with the 0.5 IoU threshold on the ScanNetV2 test set, which is 1.9% higher than the state-of-the-art method.
  arXiv  Detail & Related papers  (2022-03-28T11:22:58Z)
• Sparse Instance Activation for Real-Time Instance Segmentation [72.23597664935684]
  We propose a conceptually novel, efficient, and fully convolutional framework for real-time instance segmentation. SparseInst has extremely fast inference speed and achieves 40 FPS and 37.9 AP on the COCO benchmark.
  arXiv  Detail & Related papers  (2022-03-24T03:15:39Z)
• Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networks [64.27814530457042]
  We propose an end-to-end solution of Semantic Superpoint Tree Network (SSTNet) for proposing object instances from scene points. Key in SSTNet is an intermediate, semantic superpoint tree (SST), which is constructed based on the learned semantic features of superpoints. SSTNet ranks top on the ScanNet (V2) leaderboard, with 2% higher of mAP than the second best method.
  arXiv  Detail & Related papers  (2021-08-17T07:25:14Z)
• SOLO: A Simple Framework for Instance Segmentation [84.00519148562606]
  "instance categories" assigns categories to each pixel within an instance according to the instance's location. "SOLO" is a simple, direct, and fast framework for instance segmentation with strong performance. Our approach achieves state-of-the-art results for instance segmentation in terms of both speed and accuracy.
  arXiv  Detail & Related papers  (2021-06-30T09:56:54Z)
• PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation [111.7241018610573]
  We present PointGroup, a new end-to-end bottom-up architecture for instance segmentation. We design a two-branch network to extract point features and predict semantic labels and offsets, for shifting each point towards its respective instance centroid. A clustering component is followed to utilize both the original and offset-shifted point coordinate sets, taking advantage of their complementary strength. We conduct extensive experiments on two challenging datasets, ScanNet v2 and S3DIS, on which our method achieves the highest performance, 63.6% and 64.0%, compared to 54.9% and 54.4% achieved by former best
  arXiv  Detail & Related papers  (2020-04-03T16:26:37Z)
• 3D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentation [26.169985423367393]
  3D-MPA is a method for instance segmentation on 3D point clouds. We learn proposal features from grouped point features that voted for the same object center. A graph convolutional network introduces inter-proposal relations.
  arXiv  Detail & Related papers  (2020-03-30T23:28:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.