PV-SSD: A Multi-Modal Point Cloud Feature Fusion Method for Projection Features and Variable Receptive Field Voxel Features

• URL: http://arxiv.org/abs/2308.06791v6
• Date: Sat, 13 Apr 2024 14:39:51 GMT
• Title: PV-SSD: A Multi-Modal Point Cloud Feature Fusion Method for Projection Features and Variable Receptive Field Voxel Features
• Authors: Yongxin Shao, Aihong Tan, Zhetao Sun, Enhui Zheng, Tianhong Yan, Peng Liao,
• Abstract summary: Real-time inference from extremely sparse 3D data is a formidable challenge. To address this problem, a typical class of approaches transforms the point cloud cast into a regular data representation. This paper proposes a multi-modal point cloud feature fusion method for projection features and variable receptive field voxel features.
• Score: 1.5338480419018068
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: LiDAR-based 3D object detection and classification is crucial for autonomous driving. However, real-time inference from extremely sparse 3D data is a formidable challenge. To address this problem, a typical class of approaches transforms the point cloud cast into a regular data representation (voxels or projection maps). Then, it performs feature extraction with convolutional neural networks. However, such methods often result in a certain degree of information loss due to down-sampling or over-compression of feature information. This paper proposes a multi-modal point cloud feature fusion method for projection features and variable receptive field voxel features (PV-SSD) based on projection and variable voxelization to solve the information loss problem. We design a two-branch feature extraction structure with a 2D convolutional neural network to extract the point cloud's projection features in bird's-eye view to focus on the correlation between local features. A voxel feature extraction branch is used to extract local fine-grained features. Meanwhile, we propose a voxel feature extraction method with variable sensory fields to reduce the information loss of voxel branches due to downsampling. It avoids missing critical point information by selecting more useful feature points based on feature point weights for the detection task. In addition, we propose a multi-modal feature fusion module for point clouds. To validate the effectiveness of our method, we tested it on the KITTI dataset and ONCE dataset.

Related papers

• Self-Supervised Scene Flow Estimation with Point-Voxel Fusion and Surface Representation [30.355128117680444]
  Scene flow estimation aims to generate the 3D motion field of points between two consecutive frames of point clouds. Existing point-based methods ignore the irregularity of point clouds and have difficulty capturing long-range dependencies. We propose a point-voxel fusion method, where we utilize a voxel branch based on sparse grid attention and the shifted window strategy to capture long-range dependencies.
  arXiv  Detail & Related papers  (2024-10-17T09:05:15Z)
• PVAFN: Point-Voxel Attention Fusion Network with Multi-Pooling Enhancing for 3D Object Detection [59.355022416218624]
  integration of point and voxel representations is becoming more common in LiDAR-based 3D object detection. We propose a novel two-stage 3D object detector, called Point-Voxel Attention Fusion Network (PVAFN) PVAFN uses a multi-pooling strategy to integrate both multi-scale and region-specific information effectively.
  arXiv  Detail & Related papers  (2024-08-26T19:43:01Z)
• PV-RCNN++: Semantical Point-Voxel Feature Interaction for 3D Object Detection [22.6659359032306]
  This paper proposes a novel object detection network by semantical point-voxel feature interaction, dubbed PV-RCNN++. Experiments on the KITTI dataset show that PV-RCNN++ achieves 81.60$%$, 40.18$%$, 68.21$%$ 3D mAP on Car, Pedestrian, and Cyclist, achieving comparable or even better performance to the state-of-the-arts.
  arXiv  Detail & Related papers  (2022-08-29T08:14:00Z)
• AGO-Net: Association-Guided 3D Point Cloud Object Detection Network [86.10213302724085]
  We propose a novel 3D detection framework that associates intact features for objects via domain adaptation. We achieve new state-of-the-art performance on the KITTI 3D detection benchmark in both accuracy and speed.
  arXiv  Detail & Related papers  (2022-08-24T16:54:38Z)
• BIMS-PU: Bi-Directional and Multi-Scale Point Cloud Upsampling [60.257912103351394]
  We develop a new point cloud upsampling pipeline called BIMS-PU. We decompose the up/downsampling procedure into several up/downsampling sub-steps by breaking the target sampling factor into smaller factors. We show that our method achieves superior results to state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-06-25T13:13:37Z)
• Point Cloud Semantic Segmentation using Multi Scale Sparse Convolution Neural Network [0.0]
  We propose a feature extraction module based on multi-scale ultra-sparse convolution and a feature selection module based on channel attention. By introducing multi-scale sparse convolution, network could capture richer feature information based on convolution kernels of different sizes.
  arXiv  Detail & Related papers  (2022-05-03T15:01:20Z)
• SASA: Semantics-Augmented Set Abstraction for Point-based 3D Object Detection [78.90102636266276]
  We propose a novel set abstraction method named Semantics-Augmented Set Abstraction (SASA) Based on the estimated point-wise foreground scores, we then propose a semantics-guided point sampling algorithm to help retain more important foreground points during down-sampling. In practice, SASA shows to be effective in identifying valuable points related to foreground objects and improving feature learning for point-based 3D detection.
  arXiv  Detail & Related papers  (2022-01-06T08:54:47Z)
• InfoFocus: 3D Object Detection for Autonomous Driving with Dynamic Information Modeling [65.47126868838836]
  We propose a novel 3D object detection framework with dynamic information modeling. Coarse predictions are generated in the first stage via a voxel-based region proposal network. Experiments are conducted on the large-scale nuScenes 3D detection benchmark.
  arXiv  Detail & Related papers  (2020-07-16T18:27:08Z)
• Stereo RGB and Deeper LIDAR Based Network for 3D Object Detection [40.34710686994996]
  3D object detection has become an emerging task in autonomous driving scenarios. Previous works process 3D point clouds using either projection-based or voxel-based models. We propose the Stereo RGB and Deeper LIDAR framework which can utilize semantic and spatial information simultaneously.
  arXiv  Detail & Related papers  (2020-06-09T11:19:24Z)
• D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features [51.04841465193678]
  We leverage a 3D fully convolutional network for 3D point clouds. We propose a novel and practical learning mechanism that densely predicts both a detection score and a description feature for each 3D point. Our method achieves state-of-the-art results in both indoor and outdoor scenarios.
  arXiv  Detail & Related papers  (2020-03-06T12:51:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.