pCTFusion: Point Convolution-Transformer Fusion with Semantic Aware Loss for Outdoor LiDAR Point Cloud Segmentation

• URL: http://arxiv.org/abs/2307.14777v2
• Date: Mon, 31 Jul 2023 04:26:02 GMT
• Title: pCTFusion: Point Convolution-Transformer Fusion with Semantic Aware Loss for Outdoor LiDAR Point Cloud Segmentation
• Authors: Abhishek Kuriyal, Vaibhav Kumar, Bharat Lohani
• Abstract summary: This study proposes a new architecture, pCTFusion, which combines kernel-based convolutions and self-attention mechanisms. The proposed architecture employs two types of self-attention mechanisms, local and global, based on the hierarchical positions of the encoder blocks. The results are particularly encouraging for minor classes, often misclassified due to class imbalance, lack of space, and neighbor-aware feature encoding.
• Score: 8.24822602555667
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: LiDAR-generated point clouds are crucial for perceiving outdoor environments. The segmentation of point clouds is also essential for many applications. Previous research has focused on using self-attention and convolution (local attention) mechanisms individually in semantic segmentation architectures. However, there is limited work on combining the learned representations of these attention mechanisms to improve performance. Additionally, existing research that combines convolution with self-attention relies on global attention, which is not practical for processing large point clouds. To address these challenges, this study proposes a new architecture, pCTFusion, which combines kernel-based convolutions and self-attention mechanisms for better feature learning and capturing local and global dependencies in segmentation. The proposed architecture employs two types of self-attention mechanisms, local and global, based on the hierarchical positions of the encoder blocks. Furthermore, the existing loss functions do not consider the semantic and position-wise importance of the points, resulting in reduced accuracy, particularly at sharp class boundaries. To overcome this, the study models a novel attention-based loss function called Pointwise Geometric Anisotropy (PGA), which assigns weights based on the semantic distribution of points in a neighborhood. The proposed architecture is evaluated on SemanticKITTI outdoor dataset and showed a 5-7% improvement in performance compared to the state-of-the-art architectures. The results are particularly encouraging for minor classes, often misclassified due to class imbalance, lack of space, and neighbor-aware feature encoding. These developed methods can be leveraged for the segmentation of complex datasets and can drive real-world applications of LiDAR point cloud.

Related papers

• Global Attention-Guided Dual-Domain Point Cloud Feature Learning for Classification and Segmentation [21.421806351869552]
  We propose a Global Attention-guided Dual-domain Feature Learning network (GAD) to address the above-mentioned issues. We first devise the Contextual Position-enhanced Transformer (CPT) module, which is armed with an improved global attention mechanism. Then, the Dual-domain K-nearest neighbor Feature Fusion (DKFF) is cascaded to conduct effective feature aggregation.
  arXiv  Detail & Related papers  (2024-07-12T05:19:19Z)
• Point Tree Transformer for Point Cloud Registration [33.00645881490638]
  Point cloud registration is a fundamental task in the fields of computer vision and robotics. We propose a novel transformer-based approach for point cloud registration that efficiently extracts comprehensive local and global features. Our method achieves superior performance over the state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-06-25T13:14:26Z)
• PointJEM: Self-supervised Point Cloud Understanding for Reducing Feature Redundancy via Joint Entropy Maximization [10.53900407467811]
  We propose PointJEM, a self-supervised representation learning method applied to the point cloud field. To reduce redundant information in the features, PointJEM maximizes the joint entropy between the different parts. PointJEM achieves competitive performance in downstream tasks such as classification and segmentation.
  arXiv  Detail & Related papers  (2023-12-06T08:21:42Z)
• Bidirectional Knowledge Reconfiguration for Lightweight Point Cloud Analysis [74.00441177577295]
  Point cloud analysis faces computational system overhead, limiting its application on mobile or edge devices. This paper explores feature distillation for lightweight point cloud models. We propose bidirectional knowledge reconfiguration to distill informative contextual knowledge from the teacher to the student.
  arXiv  Detail & Related papers  (2023-10-08T11:32:50Z)
• Weakly-supervised Contrastive Learning for Unsupervised Object Discovery [52.696041556640516]
  Unsupervised object discovery is promising due to its ability to discover objects in a generic manner. We design a semantic-guided self-supervised learning model to extract high-level semantic features from images. We introduce Principal Component Analysis (PCA) to localize object regions.
  arXiv  Detail & Related papers  (2023-07-07T04:03:48Z)
• XAI for Self-supervised Clustering of Wireless Spectrum Activity [0.5809784853115825]
  We propose a methodology for explaining deep clustering, self-supervised learning architectures. For the representation learning part, our methodology employs Guided Backpropagation to interpret the regions of interest of the input data. For the clustering part, the methodology relies on Shallow Trees to explain the clustering result. Finally, a data-specific visualizations part enables connection for each of the clusters to the input data trough the relevant features.
  arXiv  Detail & Related papers  (2023-05-17T08:56:43Z)
• Centralized Feature Pyramid for Object Detection [53.501796194901964]
  Visual feature pyramid has shown its superiority in both effectiveness and efficiency in a wide range of applications. In this paper, we propose a OLO Feature Pyramid for object detection, which is based on a globally explicit centralized feature regulation.
  arXiv  Detail & Related papers  (2022-10-05T08:32:54Z)
• Inducing Gaussian Process Networks [80.40892394020797]
  We propose inducing Gaussian process networks (IGN), a simple framework for simultaneously learning the feature space as well as the inducing points. The inducing points, in particular, are learned directly in the feature space, enabling a seamless representation of complex structured domains. We report on experimental results for real-world data sets showing that IGNs provide significant advances over state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-04-21T05:27:09Z)
• PC-RGNN: Point Cloud Completion and Graph Neural Network for 3D Object Detection [57.49788100647103]
  LiDAR-based 3D object detection is an important task for autonomous driving. Current approaches suffer from sparse and partial point clouds of distant and occluded objects. In this paper, we propose a novel two-stage approach, namely PC-RGNN, dealing with such challenges by two specific solutions.
  arXiv  Detail & Related papers  (2020-12-18T18:06:43Z)
• SPU-Net: Self-Supervised Point Cloud Upsampling by Coarse-to-Fine Reconstruction with Self-Projection Optimization [52.20602782690776]
  It is expensive and tedious to obtain large scale paired sparse-canned point sets for training from real scanned sparse data. We propose a self-supervised point cloud upsampling network, named SPU-Net, to capture the inherent upsampling patterns of points lying on the underlying object surface. We conduct various experiments on both synthetic and real-scanned datasets, and the results demonstrate that we achieve comparable performance to the state-of-the-art supervised methods.
  arXiv  Detail & Related papers  (2020-12-08T14:14:09Z)
• Hybrid Multiple Attention Network for Semantic Segmentation in Aerial Images [24.35779077001839]
  We propose a novel attention-based framework named Hybrid Multiple Attention Network (HMANet) to adaptively capture global correlations. We introduce a simple yet effective region shuffle attention (RSA) module to reduce feature redundant and improve the efficiency of self-attention mechanism.
  arXiv  Detail & Related papers  (2020-01-09T07:47:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.