Bi-Directional Attention for Joint Instance and Semantic Segmentation in Point Clouds

• URL: http://arxiv.org/abs/2003.05420v1
• Date: Wed, 11 Mar 2020 17:16:07 GMT
• Title: Bi-Directional Attention for Joint Instance and Semantic Segmentation in Point Clouds
• Authors: Guangnan Wu and Zhiyi Pan and Peng Jiang and Changhe Tu
• Abstract summary: We build a Bi-Directional Attention module on backbone neural networks for 3D point cloud perception. It uses similarity matrix measured from features for one task to help aggregate non-local information for the other task. From comprehensive experiments and ablation studies on the S3DIS dataset and the PartNet dataset, the superiority of our method is verified.
• Score: 9.434847591440485
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Instance segmentation in point clouds is one of the most fine-grained ways to understand the 3D scene. Due to its close relationship to semantic segmentation, many works approach these two tasks simultaneously and leverage the benefits of multi-task learning. However, most of them only considered simple strategies such as element-wise feature fusion, which may not lead to mutual promotion. In this work, we build a Bi-Directional Attention module on backbone neural networks for 3D point cloud perception, which uses similarity matrix measured from features for one task to help aggregate non-local information for the other task, avoiding the potential feature exclusion and task conflict. From comprehensive experiments and ablation studies on the S3DIS dataset and the PartNet dataset, the superiority of our method is verified. Moreover, the mechanism of how bi-directional attention module helps joint instance and semantic segmentation is also analyzed.

Related papers

• LiSD: An Efficient Multi-Task Learning Framework for LiDAR Segmentation and Detection [6.813145466843275]
  LiSD is a voxel-based encoder-decoder framework that addresses both segmentation and detection tasks. It achieves the state-of-the-art performance of 83.3% mIoU on the nuScenes segmentation benchmark for lidar-only methods.
  arXiv  Detail & Related papers  (2024-06-11T07:26:54Z)
• Few-Shot Learning for Annotation-Efficient Nucleus Instance Segmentation [50.407071700154674]
  We propose to formulate annotation-efficient nucleus instance segmentation from the perspective of few-shot learning (FSL) Our work was motivated by that, with the prosperity of computational pathology, an increasing number of fully-annotated datasets are publicly accessible. Extensive experiments on a couple of publicly accessible datasets demonstrate that SGFSIS can outperform other annotation-efficient learning baselines.
  arXiv  Detail & Related papers  (2024-02-26T03:49:18Z)
• Lidar Panoptic Segmentation and Tracking without Bells and Whistles [48.078270195629415]
  We propose a detection-centric network for lidar segmentation and tracking. One of the core components of our network is the object instance detection branch. We evaluate our method on several 3D/4D LPS benchmarks and observe that our model establishes a new state-of-the-art among open-sourced models.
  arXiv  Detail & Related papers  (2023-10-19T04:44:43Z)
• AIMS: All-Inclusive Multi-Level Segmentation [93.5041381700744]
  We propose a new task, All-Inclusive Multi-Level (AIMS), which segments visual regions into three levels: part, entity, and relation. We also build a unified AIMS model through multi-dataset multi-task training to address the two major challenges of annotation inconsistency and task correlation.
  arXiv  Detail & Related papers  (2023-05-28T16:28:49Z)
• 3DMODT: Attention-Guided Affinities for Joint Detection & Tracking in 3D Point Clouds [95.54285993019843]
  We propose a method for joint detection and tracking of multiple objects in 3D point clouds. Our model exploits temporal information employing multiple frames to detect objects and track them in a single network.
  arXiv  Detail & Related papers  (2022-11-01T20:59:38Z)
• Semantic Segmentation for Real Point Cloud Scenes via Bilateral Augmentation and Adaptive Fusion [38.05362492645094]
  Real point cloud scenes can intuitively capture complex surroundings in the real world, but due to 3D data's raw nature, it is very challenging for machine perception. We concentrate on the essential visual task, semantic segmentation, for large-scale point cloud data collected in reality. By comparing with state-of-the-art networks on three different benchmarks, we demonstrate the effectiveness of our network.
  arXiv  Detail & Related papers  (2021-03-12T04:13:20Z)
• Improving Point Cloud Semantic Segmentation by Learning 3D Object Detection [102.62963605429508]
  Point cloud semantic segmentation plays an essential role in autonomous driving. Current 3D semantic segmentation networks focus on convolutional architectures that perform great for well represented classes. We propose a novel Aware 3D Semantic Detection (DASS) framework that explicitly leverages localization features from an auxiliary 3D object detection task.
  arXiv  Detail & Related papers  (2020-09-22T14:17:40Z)
• Self-Prediction for Joint Instance and Semantic Segmentation of Point Clouds [41.75579185647845]
  We develop a novel learning scheme named Self-Prediction for 3D instance and semantic segmentation of point clouds. Our method achieves state-of-the-art instance segmentation results on S3DIS and comparable semantic segmentation results on S3DIS and ShapeNet.
  arXiv  Detail & Related papers  (2020-07-27T07:58:00Z)
• JSENet: Joint Semantic Segmentation and Edge Detection Network for 3D Point Clouds [37.703770427574476]
  In this paper, we tackle the 3D semantic edge detection task for the first time. We present a new two-stream fully-convolutional network that jointly performs the two tasks. In particular, we design a joint refinement module that explicitly wires region information and edge information to improve the performances of both tasks.
  arXiv  Detail & Related papers  (2020-07-14T08:00:35Z)
• Few-shot 3D Point Cloud Semantic Segmentation [138.80825169240302]
  We propose a novel attention-aware multi-prototype transductive few-shot point cloud semantic segmentation method. Our proposed method shows significant and consistent improvements compared to baselines in different few-shot point cloud semantic segmentation settings.
  arXiv  Detail & Related papers  (2020-06-22T08:05:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.