Efficient 3D Perception on Multi-Sweep Point Cloud with Gumbel Spatial Pruning

• URL: http://arxiv.org/abs/2411.07742v3
• Date: Thu, 20 Feb 2025 07:08:30 GMT
• Title: Efficient 3D Perception on Multi-Sweep Point Cloud with Gumbel Spatial Pruning
• Authors: Tianyu Sun, Jianhao Li, Xueqian Zhang, Zhongdao Wang, Bailan Feng, Hengshuang Zhao,
• Abstract summary: Existing methods face limitations in recognizing objects located at a distance or occluded, due to the sparse nature of outdoor point clouds. In this work, we observe a significant mitigation of this problem by accumulating multiple temporally consecutive point cloud sweeps. We introduce a simple yet effective Gumbel Spatial Pruning layer that dynamically prunes points based on a learned end-to-end sampling.
• Score: 31.70820822331813
• License:
• Abstract: This paper studies point cloud perception within outdoor environments. Existing methods face limitations in recognizing objects located at a distance or occluded, due to the sparse nature of outdoor point clouds. In this work, we observe a significant mitigation of this problem by accumulating multiple temporally consecutive point cloud sweeps, resulting in a remarkable improvement in perception accuracy. However, the computation cost also increases, hindering previous approaches from utilizing a large number of point cloud sweeps. To tackle this challenge, we find that a considerable portion of points in the accumulated point cloud is redundant, and discarding these points has minimal impact on perception accuracy. We introduce a simple yet effective Gumbel Spatial Pruning (GSP) layer that dynamically prunes points based on a learned end-to-end sampling. The GSP layer is decoupled from other network components and thus can be seamlessly integrated into existing point cloud network architectures. Without incurring additional computational overhead, we increase the number of point cloud sweeps from 10, a common practice, to as many as 40. Consequently, there is a significant enhancement in perception performance. For instance, in nuScenes 3D object detection and BEV map segmentation tasks, our pruning strategy improves several 3D perception baseline methods.

Related papers

• PIVOT-Net: Heterogeneous Point-Voxel-Tree-based Framework for Point Cloud Compression [8.778300313732027]
  We propose a heterogeneous point cloud compression (PCC) framework. We unify typical point cloud representations -- point-based, voxel-based, and tree-based representations -- and their associated backbones. We augment the framework with a proposed context-aware upsampling for decoding and an enhanced voxel transformer for feature aggregation.
  arXiv  Detail & Related papers  (2024-02-11T16:57:08Z)
• Arbitrary point cloud upsampling via Dual Back-Projection Network [12.344557879284219]
  We propose a Dual Back-Projection network for point cloud upsampling (DBPnet) A Dual Back-Projection is formulated in an up-down-up manner for point cloud upsampling. Experimental results show that the proposed method achieves the lowest point set matching losses.
  arXiv  Detail & Related papers  (2023-07-18T06:11:09Z)
• PointCA: Evaluating the Robustness of 3D Point Cloud Completion Models Against Adversarial Examples [63.84378007819262]
  We propose PointCA, the first adversarial attack against 3D point cloud completion models. PointCA can generate adversarial point clouds that maintain high similarity with the original ones. We show that PointCA can cause a performance degradation from 77.9% to 16.7%, with the structure chamfer distance kept below 0.01.
  arXiv  Detail & Related papers  (2022-11-22T14:15:41Z)
• Shape-invariant 3D Adversarial Point Clouds [111.72163188681807]
  Adversary and invisibility are two fundamental but conflict characters of adversarial perturbations. Previous adversarial attacks on 3D point cloud recognition have often been criticized for their noticeable point outliers. We propose a novel Point-Cloud Sensitivity Map to boost both the efficiency and imperceptibility of point perturbations.
  arXiv  Detail & Related papers  (2022-03-08T12:21:35Z)
• A Conditional Point Diffusion-Refinement Paradigm for 3D Point Cloud Completion [69.32451612060214]
  Real-scanned 3D point clouds are often incomplete, and it is important to recover complete point clouds for downstream applications. Most existing point cloud completion methods use Chamfer Distance (CD) loss for training. We propose a novel Point Diffusion-Refinement (PDR) paradigm for point cloud completion.
  arXiv  Detail & Related papers  (2021-12-07T06:59:06Z)
• Deep Point Cloud Reconstruction [74.694733918351]
  Point cloud obtained from 3D scanning is often sparse, noisy, and irregular. To cope with these issues, recent studies have been separately conducted to densify, denoise, and complete inaccurate point cloud. We propose a deep point cloud reconstruction network consisting of two stages: 1) a 3D sparse stacked-hourglass network as for the initial densification and denoising, 2) a refinement via transformers converting the discrete voxels into 3D points.
  arXiv  Detail & Related papers  (2021-11-23T07:53:28Z)
• SSPU-Net: Self-Supervised Point Cloud Upsampling via Differentiable Rendering [21.563862632172363]
  We propose a self-supervised point cloud upsampling network (SSPU-Net) to generate dense point clouds without using ground truth. To achieve this, we exploit the consistency between the input sparse point cloud and generated dense point cloud for the shapes and rendered images.
  arXiv  Detail & Related papers  (2021-08-01T13:26:01Z)
• Pseudo-LiDAR Point Cloud Interpolation Based on 3D Motion Representation and Spatial Supervision [68.35777836993212]
  We propose a Pseudo-LiDAR point cloud network to generate temporally and spatially high-quality point cloud sequences. By exploiting the scene flow between point clouds, the proposed network is able to learn a more accurate representation of the 3D spatial motion relationship.
  arXiv  Detail & Related papers  (2020-06-20T03:11:04Z)
• GRNet: Gridding Residual Network for Dense Point Cloud Completion [54.43648460932248]
  Estimating the complete 3D point cloud from an incomplete one is a key problem in many vision and robotics applications. We propose a novel Gridding Residual Network (GRNet) for point cloud completion. Experimental results indicate that the proposed GRNet performs favorably against state-of-the-art methods on the ShapeNet, Completion3D, and KITTI benchmarks.
  arXiv  Detail & Related papers  (2020-06-06T02:46:39Z)
• MNEW: Multi-domain Neighborhood Embedding and Weighting for Sparse Point Clouds Segmentation [1.2380933178502298]
  We propose MNEW, including multi-domain neighborhood embedding, and attention weighting based on their geometry distance, feature similarity, and neighborhood sparsity. MNEW achieves the top performance for sparse point clouds, which is important to the application of LiDAR-based automated driving perception.
  arXiv  Detail & Related papers  (2020-04-05T18:02:07Z)
• PF-Net: Point Fractal Network for 3D Point Cloud Completion [6.504317278066694]
  Point Fractal Network (PF-Net) is a novel learning-based approach for precise and high-fidelity point cloud completion. PF-Net preserves the spatial arrangements of the incomplete point cloud and can figure out the detailed geometrical structure of the missing region(s) in the prediction. Our experiments demonstrate the effectiveness of our method for several challenging point cloud completion tasks.
  arXiv  Detail & Related papers  (2020-03-01T05:40:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.