Retro-FPN: Retrospective Feature Pyramid Network for Point Cloud Semantic Segmentation

• URL: http://arxiv.org/abs/2308.09314v1
• Date: Fri, 18 Aug 2023 05:28:25 GMT
• Title: Retro-FPN: Retrospective Feature Pyramid Network for Point Cloud Semantic Segmentation
• Authors: Peng Xiang, Xin Wen, Yu-Shen Liu, Hui Zhang, Yi Fang, Zhizhong Han
• Abstract summary: We propose Retro-FPN to model the per-point feature prediction as an explicit and retrospective refining process. Its key novelty is a retro-transformer for summarizing semantic contexts from the previous layer. We show that Retro-FPN can significantly improve performance over state-of-the-art backbones.
• Score: 65.78483246139888
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning per-point semantic features from the hierarchical feature pyramid is essential for point cloud semantic segmentation. However, most previous methods suffered from ambiguous region features or failed to refine per-point features effectively, which leads to information loss and ambiguous semantic identification. To resolve this, we propose Retro-FPN to model the per-point feature prediction as an explicit and retrospective refining process, which goes through all the pyramid layers to extract semantic features explicitly for each point. Its key novelty is a retro-transformer for summarizing semantic contexts from the previous layer and accordingly refining the features in the current stage. In this way, the categorization of each point is conditioned on its local semantic pattern. Specifically, the retro-transformer consists of a local cross-attention block and a semantic gate unit. The cross-attention serves to summarize the semantic pattern retrospectively from the previous layer. And the gate unit carefully incorporates the summarized contexts and refines the current semantic features. Retro-FPN is a pluggable neural network that applies to hierarchical decoders. By integrating Retro-FPN with three representative backbones, including both point-based and voxel-based methods, we show that Retro-FPN can significantly improve performance over state-of-the-art backbones. Comprehensive experiments on widely used benchmarks can justify the effectiveness of our design. The source is available at https://github.com/AllenXiangX/Retro-FPN

Related papers

• Position-Guided Point Cloud Panoptic Segmentation Transformer [118.17651196656178]
  This work begins by applying this appealing paradigm to LiDAR-based point cloud segmentation and obtains a simple yet effective baseline. We observe that instances in the sparse point clouds are relatively small to the whole scene and often have similar geometry but lack distinctive appearance for segmentation, which are rare in the image domain. The method, named Position-guided Point cloud Panoptic segmentation transFormer (P3Former), outperforms previous state-of-the-art methods by 3.4% and 1.2% on Semantic KITTI and nuScenes benchmark, respectively.
  arXiv  Detail & Related papers  (2023-03-23T17:59:02Z)
• Feature-Proxy Transformer for Few-Shot Segmentation [35.85575258482071]
  Few-shot segmentation (FSS) aims at performing semantic segmentation on novel classes given a few annotated support samples. This paper proposes a novel Feature- Proxy Transformer (FPTrans) method, in which the "proxy" is the vector representing a semantic class in the linear classification head. Although the framework is straightforward, we show that FPTrans achieves competitive FSS accuracy on par with state-of-the-art decoder-based methods.
  arXiv  Detail & Related papers  (2022-10-13T11:22:27Z)
• SemAffiNet: Semantic-Affine Transformation for Point Cloud Segmentation [94.11915008006483]
  We propose SemAffiNet for point cloud semantic segmentation. We conduct extensive experiments on the ScanNetV2 and NYUv2 datasets.
  arXiv  Detail & Related papers  (2022-05-26T17:00:23Z)
• SE(3)-Equivariant Attention Networks for Shape Reconstruction in Function Space [50.14426188851305]
  We propose the first SE(3)-equivariant coordinate-based network for learning occupancy fields from point clouds. In contrast to previous shape reconstruction methods that align the input to a regular grid, we operate directly on the irregular, unoriented point cloud. We show that our method outperforms previous SO(3)-equivariant methods, as well as non-equivariant methods trained on SO(3)-augmented datasets.
  arXiv  Detail & Related papers  (2022-04-05T17:59:15Z)
• REGTR: End-to-end Point Cloud Correspondences with Transformers [79.52112840465558]
  We conjecture that attention mechanisms can replace the role of explicit feature matching and RANSAC. We propose an end-to-end framework to directly predict the final set of correspondences. Our approach achieves state-of-the-art performance on 3DMatch and ModelNet benchmarks.
  arXiv  Detail & Related papers  (2022-03-28T06:01:00Z)
• Point Cloud Registration using Representative Overlapping Points [10.843159482657303]
  We propose ROPNet, a new deep learning model using Representative Overlapping Points with discriminative features for registration. Specifically, we propose a context-guided module which uses an encoder to extract global features for predicting point overlap score. Experiments over ModelNet40 using noisy and partially overlapping point clouds show that the proposed method outperforms traditional and learning-based methods.
  arXiv  Detail & Related papers  (2021-07-06T12:52:22Z)
• A^2-FPN: Attention Aggregation based Feature Pyramid Network for Instance Segmentation [68.10621089649486]
  We propose Attention Aggregation based Feature Pyramid Network (A2-FPN) to improve multi-scale feature learning. A2-FPN achieves an improvement of 2.0% and 1.4% mask AP when integrated into the strong baselines such as Cascade Mask R-CNN and Hybrid Task Cascade.
  arXiv  Detail & Related papers  (2021-05-07T11:51:08Z)
• OMNet: Learning Overlapping Mask for Partial-to-Partial Point Cloud Registration [31.108056345511976]
  OMNet is a global feature based iterative network for partial-to-partial point cloud registration. We learn masks in a coarse-to-fine manner to reject non-overlapping regions, which converting the partial-to-partial registration to the registration of the same shapes.
  arXiv  Detail & Related papers  (2021-03-01T11:59:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.