SASO: Joint 3D Semantic-Instance Segmentation via Multi-scale Semantic Association and Salient Point Clustering Optimization

• URL: http://arxiv.org/abs/2006.15015v1
• Date: Thu, 25 Jun 2020 08:55:25 GMT
• Title: SASO: Joint 3D Semantic-Instance Segmentation via Multi-scale Semantic Association and Salient Point Clustering Optimization
• Authors: Jingang Tan, Lili Chen, Kangru Wang, Jingquan Peng, Jiamao Li, Xiaolin Zhang
• Abstract summary: We propose a novel 3D point cloud segmentation framework named SASO, which jointly performs semantic and instance segmentation tasks. For semantic segmentation task, inspired by the inherent correlation among objects in spatial context, we propose a Multi-scale Semantic Association (MSA) module. For instance segmentation task, different from previous works that utilize clustering only in inference procedure, we propose a Salient Point Clustering Optimization (SPCO) module.
• Score: 8.519716460338518
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a novel 3D point cloud segmentation framework named SASO, which jointly performs semantic and instance segmentation tasks. For semantic segmentation task, inspired by the inherent correlation among objects in spatial context, we propose a Multi-scale Semantic Association (MSA) module to explore the constructive effects of the semantic context information. For instance segmentation task, different from previous works that utilize clustering only in inference procedure, we propose a Salient Point Clustering Optimization (SPCO) module to introduce a clustering procedure into the training process and impel the network focusing on points that are difficult to be distinguished. In addition, because of the inherent structures of indoor scenes, the imbalance problem of the category distribution is rarely considered but severely limits the performance of 3D scene perception. To address this issue, we introduce an adaptive Water Filling Sampling (WFS) algorithm to balance the category distribution of training data. Extensive experiments demonstrate that our method outperforms the state-of-the-art methods on benchmark datasets in both semantic segmentation and instance segmentation tasks.

Related papers

• P2Seg: Pointly-supervised Segmentation via Mutual Distillation [23.979786026101024]
  We develop a Mutual Distillation Module (MDM) to leverage the complementary strengths of both instance position and semantic information. Our method achieves 55.7 mAP$_50$ and 17.6 mAP on the PASCAL VOC and MS COCO datasets.
  arXiv  Detail & Related papers  (2024-01-18T03:41:38Z)
• CSL: Class-Agnostic Structure-Constrained Learning for Segmentation Including the Unseen [62.72636247006293]
  Class-Agnostic Structure-Constrained Learning is a plug-in framework that can integrate with existing methods. We propose soft assignment and mask split methodologies that enhance OOD object segmentation. Empirical evaluations demonstrate CSL's prowess in boosting the performance of existing algorithms spanning OOD segmentation, ZS3, and DA segmentation.
  arXiv  Detail & Related papers  (2023-12-09T11:06:18Z)
• Towards accurate instance segmentation in large-scale LiDAR point clouds [17.808580509435565]
  Panoptic segmentation is the combination of semantic and instance segmentation. This study explores the steps of the panoptic segmentation pipeline concerned with clustering points into object instances. We find that a carefully designed clustering strategy, which leverages multiple types of learned point embeddings, significantly improves instance segmentation.
  arXiv  Detail & Related papers  (2023-07-06T09:29:03Z)
• AIMS: All-Inclusive Multi-Level Segmentation [93.5041381700744]
  We propose a new task, All-Inclusive Multi-Level (AIMS), which segments visual regions into three levels: part, entity, and relation. We also build a unified AIMS model through multi-dataset multi-task training to address the two major challenges of annotation inconsistency and task correlation.
  arXiv  Detail & Related papers  (2023-05-28T16:28:49Z)
• Semantic Segmentation-Assisted Instance Feature Fusion for Multi-Level 3D Part Instance Segmentation [17.929866369256555]
  We present a new method for 3D part instance segmentation. Our method exploits semantic segmentation to fuse nonlocal instance features, such as center prediction. Our method outperforms existing methods with a large-margin improvement in the PartNet benchmark.
  arXiv  Detail & Related papers  (2022-08-09T13:22:55Z)
• Cluster-to-adapt: Few Shot Domain Adaptation for Semantic Segmentation across Disjoint Labels [80.05697343811893]
  Cluster-to-Adapt (C2A) is a computationally efficient clustering-based approach for domain adaptation across segmentation datasets. We show that such a clustering objective enforced in a transformed feature space serves to automatically select categories across source and target domains.
  arXiv  Detail & Related papers  (2022-08-04T17:57:52Z)
• PointInst3D: Segmenting 3D Instances by Points [136.7261709896713]
  We propose a fully-convolutional 3D point cloud instance segmentation method that works in a per-point prediction fashion. We find the key to its success is assigning a suitable target to each sampled point. Our approach achieves promising results on both ScanNet and S3DIS benchmarks.
  arXiv  Detail & Related papers  (2022-04-25T02:41:46Z)
• Improving Semi-Supervised and Domain-Adaptive Semantic Segmentation with Self-Supervised Depth Estimation [94.16816278191477]
  We present a framework for semi-adaptive and domain-supervised semantic segmentation. It is enhanced by self-supervised monocular depth estimation trained only on unlabeled image sequences. We validate the proposed model on the Cityscapes dataset.
  arXiv  Detail & Related papers  (2021-08-28T01:33:38Z)
• Semantic Attention and Scale Complementary Network for Instance Segmentation in Remote Sensing Images [54.08240004593062]
  We propose an end-to-end multi-category instance segmentation model, which consists of a Semantic Attention (SEA) module and a Scale Complementary Mask Branch (SCMB) SEA module contains a simple fully convolutional semantic segmentation branch with extra supervision to strengthen the activation of interest instances on the feature map. SCMB extends the original single mask branch to trident mask branches and introduces complementary mask supervision at different scales.
  arXiv  Detail & Related papers  (2021-07-25T08:53:59Z)
• Instance segmentation with the number of clusters incorporated in embedding learning [15.054120734581826]
  We propose to embed clustering information into an embedding learning framework FCRNet. FCRNet relieves the complexity of post process by incorporating the number of clustering groups into the embedding space. The superior performance of FCRNet is verified and compared with other methods on the nucleus dataset BBBC006.
  arXiv  Detail & Related papers  (2021-03-27T10:03:19Z)
• Self-Prediction for Joint Instance and Semantic Segmentation of Point Clouds [41.75579185647845]
  We develop a novel learning scheme named Self-Prediction for 3D instance and semantic segmentation of point clouds. Our method achieves state-of-the-art instance segmentation results on S3DIS and comparable semantic segmentation results on S3DIS and ShapeNet.
  arXiv  Detail & Related papers  (2020-07-27T07:58:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.