Region-Enhanced Feature Learning for Scene Semantic Segmentation

• URL: http://arxiv.org/abs/2304.07486v3
• Date: Wed, 17 Jan 2024 02:58:20 GMT
• Title: Region-Enhanced Feature Learning for Scene Semantic Segmentation
• Authors: Xin Kang, Chaoqun Wang, Xuejin Chen
• Abstract summary: We propose using regions as the intermediate representation of point clouds instead of fine-grained points or voxels to reduce the computational burden. We design a region-based feature enhancement (RFE) module, which consists of a Semantic-Spatial Region Extraction stage and a Region Dependency Modeling stage. Our REFL-Net achieves 1.8% mIoU gain on ScanNetV2 and 1.7% mIoU gain on S3DIS datasets with negligible computational cost.
• Score: 19.20735517821943
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Semantic segmentation in complex scenes relies not only on object appearance but also on object location and the surrounding environment. Nonetheless, it is difficult to model long-range context in the format of pairwise point correlations due to the huge computational cost for large-scale point clouds. In this paper, we propose using regions as the intermediate representation of point clouds instead of fine-grained points or voxels to reduce the computational burden. We introduce a novel Region-Enhanced Feature Learning Network (REFL-Net) that leverages region correlations to enhance point feature learning. We design a region-based feature enhancement (RFE) module, which consists of a Semantic-Spatial Region Extraction stage and a Region Dependency Modeling stage. In the first stage, the input points are grouped into a set of regions based on their semantic and spatial proximity. In the second stage, we explore inter-region semantic and spatial relationships by employing a self-attention block on region features and then fuse point features with the region features to obtain more discriminative representations. Our proposed RFE module is plug-and-play and can be integrated with common semantic segmentation backbones. We conduct extensive experiments on ScanNetV2 and S3DIS datasets and evaluate our RFE module with different segmentation backbones. Our REFL-Net achieves 1.8% mIoU gain on ScanNetV2 and 1.7% mIoU gain on S3DIS with negligible computational cost compared with backbone models. Both quantitative and qualitative results show the powerful long-range context modeling ability and strong generalization ability of our REFL-Net.

Related papers

• Enhanced Semantic Segmentation for Large-Scale and Imbalanced Point Clouds [6.253217784798542]
  Small-sized objects are prone to be under-sampled or misclassified due to their low occurrence frequency. We propose the Multilateral Cascading Network (MCNet) for large-scale and sample-imbalanced point cloud scenes.
  arXiv  Detail & Related papers  (2024-09-21T02:23:01Z)
• Local region-learning modules for point cloud classification [0.0]
  We present two local region-learning modules that infer the appropriate shift for each center point and alter the radius of each local region. We integrated both modules independently and together to the PointNet++ and PointCNN object classification architectures. Our experiments on ShapeNet data set showed that the modules are also effective on 3D CAD models.
  arXiv  Detail & Related papers  (2023-03-30T12:45:46Z)
• DQnet: Cross-Model Detail Querying for Camouflaged Object Detection [54.82390534024954]
  A convolutional neural network (CNN) for camouflaged object detection tends to activate local discriminative regions while ignoring complete object extent. In this paper, we argue that partial activation is caused by the intrinsic characteristics of CNN. In order to obtain feature maps that could activate full object extent, a novel framework termed Cross-Model Detail Querying network (DQnet) is proposed.
  arXiv  Detail & Related papers  (2022-12-16T06:23:58Z)
• Adaptive Edge-to-Edge Interaction Learning for Point Cloud Analysis [118.30840667784206]
  Key issue for point cloud data processing is extracting useful information from local regions. Previous works ignore the relation between edges in local regions, which encodes the local shape information. This paper proposes a novel Adaptive Edge-to-Edge Interaction Learning module.
  arXiv  Detail & Related papers  (2022-11-20T07:10:14Z)
• LACV-Net: Semantic Segmentation of Large-Scale Point Cloud Scene via Local Adaptive and Comprehensive VLAD [13.907586081922345]
  We propose an end-to-end deep neural network called LACV-Net for large-scale point cloud semantic segmentation. The proposed network contains three main components: 1) a local adaptive feature augmentation module (LAFA) to adaptively learn the similarity of centroids and neighboring points to augment the local context; 2) a comprehensive VLAD module that fuses local features with multi-layer, multi-scale, and multi-resolution to represent a comprehensive global description vector; and 3) an aggregation loss function to effectively optimize the segmentation boundaries by constraining the adaptive weight from the LAFA module.
  arXiv  Detail & Related papers  (2022-10-12T02:11:00Z)
• Refine and Represent: Region-to-Object Representation Learning [55.70715883351945]
  We present Region-to-Object Representation Learning (R2O) which unifies region-based and object-centric pretraining. R2O operates by training an encoder to dynamically refine region-based segments into object-centric masks. After pretraining on ImageNet, R2O models are able to surpass existing state-of-the-art in unsupervised object segmentation.
  arXiv  Detail & Related papers  (2022-08-25T01:44:28Z)
• PRA-Net: Point Relation-Aware Network for 3D Point Cloud Analysis [56.91758845045371]
  We propose a novel framework named Point Relation-Aware Network (PRA-Net) It is composed of an Intra-region Structure Learning (ISL) module and an Inter-region Relation Learning (IRL) module. Experiments on several 3D benchmarks covering shape classification, keypoint estimation, and part segmentation have verified the effectiveness and the ability of PRA-Net.
  arXiv  Detail & Related papers  (2021-12-09T13:24:43Z)
• Global Aggregation then Local Distribution for Scene Parsing [99.1095068574454]
  We show that our approach can be modularized as an end-to-end trainable block and easily plugged into existing semantic segmentation networks. Our approach allows us to build new state of the art on major semantic segmentation benchmarks including Cityscapes, ADE20K, Pascal Context, Camvid and COCO-stuff.
  arXiv  Detail & Related papers  (2021-07-28T03:46:57Z)
• LRC-Net: Learning Discriminative Features on Point Clouds by Encoding Local Region Contexts [65.79931333193016]
  We present a novel Local-Region-Context Network (LRC-Net) to learn discriminative features on point clouds. LRC-Net encodes fine-grained contexts inside and among local regions simultaneously. Results show LRC-Net is competitive with state-of-the-art methods in shape classification and shape segmentation applications.
  arXiv  Detail & Related papers  (2020-03-18T14:34:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.