3D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentation

• URL: http://arxiv.org/abs/2003.13867v1
• Date: Mon, 30 Mar 2020 23:28:50 GMT
• Title: 3D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentation
• Authors: Francis Engelmann, Martin Bokeloh, Alireza Fathi, Bastian Leibe, Matthias Nie{\ss}ner
• Abstract summary: 3D-MPA is a method for instance segmentation on 3D point clouds. We learn proposal features from grouped point features that voted for the same object center. A graph convolutional network introduces inter-proposal relations.
• Score: 26.169985423367393
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present 3D-MPA, a method for instance segmentation on 3D point clouds. Given an input point cloud, we propose an object-centric approach where each point votes for its object center. We sample object proposals from the predicted object centers. Then, we learn proposal features from grouped point features that voted for the same object center. A graph convolutional network introduces inter-proposal relations, providing higher-level feature learning in addition to the lower-level point features. Each proposal comprises a semantic label, a set of associated points over which we define a foreground-background mask, an objectness score and aggregation features. Previous works usually perform non-maximum-suppression (NMS) over proposals to obtain the final object detections or semantic instances. However, NMS can discard potentially correct predictions. Instead, our approach keeps all proposals and groups them together based on the learned aggregation features. We show that grouping proposals improves over NMS and outperforms previous state-of-the-art methods on the tasks of 3D object detection and semantic instance segmentation on the ScanNetV2 benchmark and the S3DIS dataset.

Related papers

• Open-Vocabulary Octree-Graph for 3D Scene Understanding [54.11828083068082]
  Octree-Graph is a novel scene representation for open-vocabulary 3D scene understanding. An adaptive-octree structure is developed that stores semantics and depicts the occupancy of an object adjustably according to its shape.
  arXiv  Detail & Related papers  (2024-11-25T10:14:10Z)
• SPGroup3D: Superpoint Grouping Network for Indoor 3D Object Detection [23.208654655032955]
  Current 3D object detection methods for indoor scenes mainly follow the voting-and-grouping strategy to generate proposals. We propose a novel superpoint grouping network for indoor anchor-free one-stage 3D object detection. Experimental results demonstrate our method achieves state-of-the-art performance on ScanNet V2, SUN RGB-D, and S3DIS datasets.
  arXiv  Detail & Related papers  (2023-12-21T08:08:02Z)
• Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidance [49.14140194332482]
  We introduce Open3DIS, a novel solution designed to tackle the problem of Open-Vocabulary Instance within 3D scenes. Objects within 3D environments exhibit diverse shapes, scales, and colors, making precise instance-level identification a challenging task.
  arXiv  Detail & Related papers  (2023-12-17T10:07:03Z)
• ProposalContrast: Unsupervised Pre-training for LiDAR-based 3D Object Detection [114.54835359657707]
  ProposalContrast is an unsupervised point cloud pre-training framework. It learns robust 3D representations by contrasting region proposals. ProposalContrast is verified on various 3D detectors.
  arXiv  Detail & Related papers  (2022-07-26T04:45:49Z)
• SASA: Semantics-Augmented Set Abstraction for Point-based 3D Object Detection [78.90102636266276]
  We propose a novel set abstraction method named Semantics-Augmented Set Abstraction (SASA) Based on the estimated point-wise foreground scores, we then propose a semantics-guided point sampling algorithm to help retain more important foreground points during down-sampling. In practice, SASA shows to be effective in identifying valuable points related to foreground objects and improving feature learning for point-based 3D detection.
  arXiv  Detail & Related papers  (2022-01-06T08:54:47Z)
• Background-Aware 3D Point Cloud Segmentationwith Dynamic Point Feature Aggregation [12.093182949686781]
  We propose a novel 3D point cloud learning network, referred to as Dynamic Point Feature Aggregation Network (DPFA-Net) DPFA-Net has two variants for semantic segmentation and classification of 3D point clouds. It achieves the state-of-the-art overall accuracy score for semantic segmentation on the S3DIS dataset.
  arXiv  Detail & Related papers  (2021-11-14T05:46:05Z)
• Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networks [64.27814530457042]
  We propose an end-to-end solution of Semantic Superpoint Tree Network (SSTNet) for proposing object instances from scene points. Key in SSTNet is an intermediate, semantic superpoint tree (SST), which is constructed based on the learned semantic features of superpoints. SSTNet ranks top on the ScanNet (V2) leaderboard, with 2% higher of mAP than the second best method.
  arXiv  Detail & Related papers  (2021-08-17T07:25:14Z)
• Hierarchical Aggregation for 3D Instance Segmentation [41.20244892803604]
  We propose a clustering-based framework named HAIS, which makes full use of spatial relation of points and point sets. It ranks 1st on the ScanNet v2 benchmark, achieving the highest 69.9% AP50 and surpassing previous state-of-the-art (SOTA) methods by a large margin.
  arXiv  Detail & Related papers  (2021-08-05T03:34:34Z)
• PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation [111.7241018610573]
  We present PointGroup, a new end-to-end bottom-up architecture for instance segmentation. We design a two-branch network to extract point features and predict semantic labels and offsets, for shifting each point towards its respective instance centroid. A clustering component is followed to utilize both the original and offset-shifted point coordinate sets, taking advantage of their complementary strength. We conduct extensive experiments on two challenging datasets, ScanNet v2 and S3DIS, on which our method achieves the highest performance, 63.6% and 64.0%, compared to 54.9% and 54.4% achieved by former best
  arXiv  Detail & Related papers  (2020-04-03T16:26:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.