PV-SSD: A Multi-Modal Point Cloud Feature Fusion Method for Projection Features and Variable Receptive Field Voxel Features

• URL: http://arxiv.org/abs/2308.06791v6
• Date: Sat, 13 Apr 2024 14:39:51 GMT
• Title: PV-SSD: A Multi-Modal Point Cloud Feature Fusion Method for Projection Features and Variable Receptive Field Voxel Features
• Authors: Yongxin Shao, Aihong Tan, Zhetao Sun, Enhui Zheng, Tianhong Yan, Peng Liao,
• Abstract summary: Real-time inference from extremely sparse 3D data is a formidable challenge. To address this problem, a typical class of approaches transforms the point cloud cast into a regular data representation. This paper proposes a multi-modal point cloud feature fusion method for projection features and variable receptive field voxel features.
• Score: 1.5338480419018068
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: LiDAR-based 3D object detection and classification is crucial for autonomous driving. However, real-time inference from extremely sparse 3D data is a formidable challenge. To address this problem, a typical class of approaches transforms the point cloud cast into a regular data representation (voxels or projection maps). Then, it performs feature extraction with convolutional neural networks. However, such methods often result in a certain degree of information loss due to down-sampling or over-compression of feature information. This paper proposes a multi-modal point cloud feature fusion method for projection features and variable receptive field voxel features (PV-SSD) based on projection and variable voxelization to solve the information loss problem. We design a two-branch feature extraction structure with a 2D convolutional neural network to extract the point cloud's projection features in bird's-eye view to focus on the correlation between local features. A voxel feature extraction branch is used to extract local fine-grained features. Meanwhile, we propose a voxel feature extraction method with variable sensory fields to reduce the information loss of voxel branches due to downsampling. It avoids missing critical point information by selecting more useful feature points based on feature point weights for the detection task. In addition, we propose a multi-modal feature fusion module for point clouds. To validate the effectiveness of our method, we tested it on the KITTI dataset and ONCE dataset.

Related papers

• PV-RCNN++: Semantical Point-Voxel Feature Interaction for 3D Object Detection [22.6659359032306]
  This paper proposes a novel object detection network by semantical point-voxel feature interaction, dubbed PV-RCNN++. Experiments on the KITTI dataset show that PV-RCNN++ achieves 81.60$%$, 40.18$%$, 68.21$%$ 3D mAP on Car, Pedestrian, and Cyclist, achieving comparable or even better performance to the state-of-the-arts.
  arXiv  Detail & Related papers  (2022-08-29T08:14:00Z)
• AGO-Net: Association-Guided 3D Point Cloud Object Detection Network [86.10213302724085]
  We propose a novel 3D detection framework that associates intact features for objects via domain adaptation. We achieve new state-of-the-art performance on the KITTI 3D detection benchmark in both accuracy and speed.
  arXiv  Detail & Related papers  (2022-08-24T16:54:38Z)
• BIMS-PU: Bi-Directional and Multi-Scale Point Cloud Upsampling [60.257912103351394]
  We develop a new point cloud upsampling pipeline called BIMS-PU. We decompose the up/downsampling procedure into several up/downsampling sub-steps by breaking the target sampling factor into smaller factors. We show that our method achieves superior results to state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-06-25T13:13:37Z)
• Point Cloud Semantic Segmentation using Multi Scale Sparse Convolution Neural Network [0.0]
  We propose a feature extraction module based on multi-scale ultra-sparse convolution and a feature selection module based on channel attention. By introducing multi-scale sparse convolution, network could capture richer feature information based on convolution kernels of different sizes.
  arXiv  Detail & Related papers  (2022-05-03T15:01:20Z)
• SASA: Semantics-Augmented Set Abstraction for Point-based 3D Object Detection [78.90102636266276]
  We propose a novel set abstraction method named Semantics-Augmented Set Abstraction (SASA) Based on the estimated point-wise foreground scores, we then propose a semantics-guided point sampling algorithm to help retain more important foreground points during down-sampling. In practice, SASA shows to be effective in identifying valuable points related to foreground objects and improving feature learning for point-based 3D detection.
  arXiv  Detail & Related papers  (2022-01-06T08:54:47Z)
• HVPR: Hybrid Voxel-Point Representation for Single-stage 3D Object Detection [39.64891219500416]
  3D object detection methods exploit either voxel-based or point-based features to represent 3D objects in a scene. We introduce in this paper a novel single-stage 3D detection method having the merit of both voxel-based and point-based features.
  arXiv  Detail & Related papers  (2021-04-02T06:34:49Z)
• PRIN/SPRIN: On Extracting Point-wise Rotation Invariant Features [91.2054994193218]
  We propose a point-set learning framework PRIN, focusing on rotation invariant feature extraction in point clouds analysis. In addition, we extend PRIN to a sparse version called SPRIN, which directly operates on sparse point clouds. Results show that, on the dataset with randomly rotated point clouds, SPRIN demonstrates better performance than state-of-the-art methods without any data augmentation.
  arXiv  Detail & Related papers  (2021-02-24T06:44:09Z)
• Spherical Interpolated Convolutional Network with Distance-Feature Density for 3D Semantic Segmentation of Point Clouds [24.85151376535356]
  Spherical interpolated convolution operator is proposed to replace the traditional grid-shaped 3D convolution operator. The proposed method achieves good performance on the ScanNet dataset and Paris-Lille-3D dataset.
  arXiv  Detail & Related papers  (2020-11-27T15:35:12Z)
• InfoFocus: 3D Object Detection for Autonomous Driving with Dynamic Information Modeling [65.47126868838836]
  We propose a novel 3D object detection framework with dynamic information modeling. Coarse predictions are generated in the first stage via a voxel-based region proposal network. Experiments are conducted on the large-scale nuScenes 3D detection benchmark.
  arXiv  Detail & Related papers  (2020-07-16T18:27:08Z)
• Stereo RGB and Deeper LIDAR Based Network for 3D Object Detection [40.34710686994996]
  3D object detection has become an emerging task in autonomous driving scenarios. Previous works process 3D point clouds using either projection-based or voxel-based models. We propose the Stereo RGB and Deeper LIDAR framework which can utilize semantic and spatial information simultaneously.
  arXiv  Detail & Related papers  (2020-06-09T11:19:24Z)
• D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features [51.04841465193678]
  We leverage a 3D fully convolutional network for 3D point clouds. We propose a novel and practical learning mechanism that densely predicts both a detection score and a description feature for each 3D point. Our method achieves state-of-the-art results in both indoor and outdoor scenarios.
  arXiv  Detail & Related papers  (2020-03-06T12:51:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.