Learning to Drop Points for LiDAR Scan Synthesis

• URL: http://arxiv.org/abs/2102.11952v1
• Date: Tue, 23 Feb 2021 21:53:14 GMT
• Title: Learning to Drop Points for LiDAR Scan Synthesis
• Authors: Kazuto Nakashima and Ryo Kurazume
• Abstract summary: Generative modeling of 3D scenes is a crucial topic for aiding mobile robots to improve unreliable observations. Most existing studies on point clouds have focused on small and uniform-density data. 3D LiDAR point clouds widely used in mobile robots are non-trivial to be handled because of the large number of points and varying-density. This paper proposes a novel framework based on generative adversarial networks to synthesize realistic LiDAR data as an improved 2D representation.
• Score: 5.132259673802809
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative modeling of 3D scenes is a crucial topic for aiding mobile robots to improve unreliable observations. However, despite the rapid progress in the natural image domain, building generative models is still challenging for 3D data, such as point clouds. Most existing studies on point clouds have focused on small and uniform-density data. In contrast, 3D LiDAR point clouds widely used in mobile robots are non-trivial to be handled because of the large number of points and varying-density. To circumvent this issue, 3D-to-2D projected representation such as a cylindrical depth map has been studied in existing LiDAR processing tasks but susceptible to discrete lossy pixels caused by failures of laser reflection. This paper proposes a novel framework based on generative adversarial networks to synthesize realistic LiDAR data as an improved 2D representation. Our generative architectures are designed to learn a distribution of inverse depth maps and simultaneously simulate the lossy pixels, which enables us to decompose an underlying smooth geometry and the corresponding uncertainty of laser reflection. To simulate the lossy pixels, we propose a differentiable framework to learn to produce sample-dependent binary masks using the Gumbel-Sigmoid reparametrization trick. We demonstrate the effectiveness of our approach in synthesis and reconstruction tasks on two LiDAR datasets. We further showcase potential applications by recovering various corruptions in LiDAR data.

Related papers

• Robust 3D Point Clouds Classification based on Declarative Defenders [18.51700931775295]
  3D point clouds are unstructured and sparse, while 2D images are structured and dense. In this paper, we explore three distinct algorithms for mapping 3D point clouds into 2D images. The proposed approaches demonstrate superior accuracy and robustness against adversarial attacks.
  arXiv  Detail & Related papers  (2024-10-13T01:32:38Z)
• Approaching Outside: Scaling Unsupervised 3D Object Detection from 2D Scene [22.297964850282177]
  We propose LiDAR-2D Self-paced Learning (LiSe) for unsupervised 3D detection. RGB images serve as a valuable complement to LiDAR data, offering precise 2D localization cues. Our framework devises a self-paced learning pipeline that incorporates adaptive sampling and weak model aggregation strategies.
  arXiv  Detail & Related papers  (2024-07-11T14:58:49Z)
• VFMM3D: Releasing the Potential of Image by Vision Foundation Model for Monocular 3D Object Detection [80.62052650370416]
  monocular 3D object detection holds significant importance across various applications, including autonomous driving and robotics. In this paper, we present VFMM3D, an innovative framework that leverages the capabilities of Vision Foundation Models (VFMs) to accurately transform single-view images into LiDAR point cloud representations.
  arXiv  Detail & Related papers  (2024-04-15T03:12:12Z)
• TULIP: Transformer for Upsampling of LiDAR Point Clouds [32.77657816997911]
  LiDAR Up is a challenging task for the perception systems of robots and autonomous vehicles. Recent works propose to solve this problem by converting LiDAR data from 3D Euclidean space into an image super-resolution problem in 2D image space. We propose T geometries, a new method to reconstruct high-resolution LiDAR point clouds from low-resolution LiDAR input.
  arXiv  Detail & Related papers  (2023-12-11T10:43:28Z)
• StableDreamer: Taming Noisy Score Distillation Sampling for Text-to-3D [88.66678730537777]
  We present StableDreamer, a methodology incorporating three advances. First, we formalize the equivalence of the SDS generative prior and a simple supervised L2 reconstruction loss. Second, our analysis shows that while image-space diffusion contributes to geometric precision, latent-space diffusion is crucial for vivid color rendition.
  arXiv  Detail & Related papers  (2023-12-02T02:27:58Z)
• Generative Range Imaging for Learning Scene Priors of 3D LiDAR Data [3.9447103367861542]
  This paper proposes a generative model of LiDAR range images applicable to the data-level domain transfer. Motivated by the fact that LiDAR measurement is based on point-by-point range imaging, we train an implicit image representation-based generative adversarial networks. We demonstrate the fidelity and diversity of our model in comparison with the point-based and image-based state-of-the-art generative models.
  arXiv  Detail & Related papers  (2022-10-21T06:08:39Z)
• RiCS: A 2D Self-Occlusion Map for Harmonizing Volumetric Objects [68.85305626324694]
  Ray-marching in Camera Space (RiCS) is a new method to represent the self-occlusions of foreground objects in 3D into a 2D self-occlusion map. We show that our representation map not only allows us to enhance the image quality but also to model temporally coherent complex shadow effects.
  arXiv  Detail & Related papers  (2022-05-14T05:35:35Z)
• Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based Perception [122.53774221136193]
  State-of-the-art methods for driving-scene LiDAR-based perception often project the point clouds to 2D space and then process them via 2D convolution. A natural remedy is to utilize the 3D voxelization and 3D convolution network. We propose a new framework for the outdoor LiDAR segmentation, where cylindrical partition and asymmetrical 3D convolution networks are designed to explore the 3D geometric pattern.
  arXiv  Detail & Related papers  (2021-09-12T06:25:11Z)
• RandomRooms: Unsupervised Pre-training from Synthetic Shapes and Randomized Layouts for 3D Object Detection [138.2892824662943]
  A promising solution is to make better use of the synthetic dataset, which consists of CAD object models, to boost the learning on real datasets. Recent work on 3D pre-training exhibits failure when transfer features learned on synthetic objects to other real-world applications. In this work, we put forward a new method called RandomRooms to accomplish this objective.
  arXiv  Detail & Related papers  (2021-08-17T17:56:12Z)
• Secrets of 3D Implicit Object Shape Reconstruction in the Wild [92.5554695397653]
  Reconstructing high-fidelity 3D objects from sparse, partial observation is crucial for various applications in computer vision, robotics, and graphics. Recent neural implicit modeling methods show promising results on synthetic or dense datasets. But, they perform poorly on real-world data that is sparse and noisy. This paper analyzes the root cause of such deficient performance of a popular neural implicit model.
  arXiv  Detail & Related papers  (2021-01-18T03:24:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.