Semi-supervised Implicit Scene Completion from Sparse LiDAR

• URL: http://arxiv.org/abs/2111.14798v1
• Date: Mon, 29 Nov 2021 18:50:09 GMT
• Title: Semi-supervised Implicit Scene Completion from Sparse LiDAR
• Authors: Pengfei Li, Yongliang Shi, Tianyu Liu, Hao Zhao, Guyue Zhou, Ya-Qin Zhang
• Abstract summary: We develop a novel formulation that conditions the semi-supervised implicit function on localized shape embeddings. It exploits the strong representation learning power of sparse convolutional networks to generate shape-aware dense feature volumes. We demonstrate intrinsic properties of this new learning system and its usefulness in real-world road scenes.
• Score: 11.136332180451308
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Recent advances show that semi-supervised implicit representation learning can be achieved through physical constraints like Eikonal equations. However, this scheme has not yet been successfully used for LiDAR point cloud data, due to its spatially varying sparsity. In this paper, we develop a novel formulation that conditions the semi-supervised implicit function on localized shape embeddings. It exploits the strong representation learning power of sparse convolutional networks to generate shape-aware dense feature volumes, while still allows semi-supervised signed distance function learning without knowing its exact values at free space. With extensive quantitative and qualitative results, we demonstrate intrinsic properties of this new learning system and its usefulness in real-world road scenes. Notably, we improve IoU from 26.3% to 51.0% on SemanticKITTI. Moreover, we explore two paradigms to integrate semantic label predictions, achieving implicit semantic completion. Code and models can be accessed at https://github.com/OPEN-AIR-SUN/SISC.

Related papers

• SN-LiDAR: Semantic Neural Fields for Novel Space-time View LiDAR Synthesis [11.615282010184917]
  We propose SN-LiDAR, a method that jointly performs accurate semantic segmentation, high-quality geometric reconstruction, and realistic LiDAR synthesis. Specifically, we employ a coarse-to-fine planar-grid feature representation to extract global features from multi-frame point clouds. Experiments on Semantic KITTI and KITTI-360 demonstrate the superiority of SN-LiDAR in both semantic and geometric reconstruction.
  arXiv  Detail & Related papers  (2025-04-11T08:51:23Z)
• SuperFlow++: Enhanced Spatiotemporal Consistency for Cross-Modal Data Pretraining [62.433137130087445]
  SuperFlow++ is a novel framework that integrates pretraining and downstream tasks using consecutive camera pairs. We show that SuperFlow++ outperforms state-of-the-art methods across diverse tasks and driving conditions. With strong generalizability and computational efficiency, SuperFlow++ establishes a new benchmark for data-efficient LiDAR-based perception in autonomous driving.
  arXiv  Detail & Related papers  (2025-03-25T17:59:57Z)
• Empowering Sparse-Input Neural Radiance Fields with Dual-Level Semantic Guidance from Dense Novel Views [66.1245505423179]
  We show that rendered semantics can be treated as a more robust form of augmented data than rendered RGB. Our method enhances NeRF's performance by incorporating guidance derived from the rendered semantics.
  arXiv  Detail & Related papers  (2025-03-04T03:13:44Z)
• 4D Contrastive Superflows are Dense 3D Representation Learners [62.433137130087445]
  We introduce SuperFlow, a novel framework designed to harness consecutive LiDAR-camera pairs for establishing pretraining objectives. To further boost learning efficiency, we incorporate a plug-and-play view consistency module that enhances alignment of the knowledge distilled from camera views.
  arXiv  Detail & Related papers  (2024-07-08T17:59:54Z)
• IPoD: Implicit Field Learning with Point Diffusion for Generalizable 3D Object Reconstruction from Single RGB-D Images [50.4538089115248]
  Generalizable 3D object reconstruction from single-view RGB-D images remains a challenging task. We propose a novel approach, IPoD, which harmonizes implicit field learning with point diffusion. Experiments conducted on the CO3D-v2 dataset affirm the superiority of IPoD, achieving 7.8% improvement in F-score and 28.6% in Chamfer distance over existing methods.
  arXiv  Detail & Related papers  (2024-03-30T07:17:37Z)
• 2D Feature Distillation for Weakly- and Semi-Supervised 3D Semantic Segmentation [92.17700318483745]
  We propose an image-guidance network (IGNet) which builds upon the idea of distilling high level feature information from a domain adapted synthetically trained 2D semantic segmentation network. IGNet achieves state-of-the-art results for weakly-supervised LiDAR semantic segmentation on ScribbleKITTI, boasting up to 98% relative performance to fully supervised training with only 8% labeled points.
  arXiv  Detail & Related papers  (2023-11-27T07:57:29Z)
• Attention-Guided Lidar Segmentation and Odometry Using Image-to-Point Cloud Saliency Transfer [6.058427379240697]
  SalLiDAR is a saliency-guided 3D semantic segmentation model that integrates saliency information to improve segmentation performance. SalLONet is a self-supervised saliency-guided LiDAR odometry network that uses the semantic and saliency predictions of SalLiDAR to achieve better odometry estimation.
  arXiv  Detail & Related papers  (2023-08-28T06:22:10Z)
• LODE: Locally Conditioned Eikonal Implicit Scene Completion from Sparse LiDAR [5.900616958195897]
  Scene completion refers to obtaining dense scene representation from an incomplete perception of complex 3D scenes. Recent advances show that implicit representation learning can be leveraged for continuous scene completion. We propose a novel Eikonal formulation that conditions the implicit representation on localized shape priors which function as dense boundary value constraints.
  arXiv  Detail & Related papers  (2023-02-27T18:59:58Z)
• Few 'Zero Level Set'-Shot Learning of Shape Signed Distance Functions in Feature Space [6.675491069288519]
  We explore a new idea for learning based shape reconstruction from a point cloud. We use a convolutional encoder to build a feature space given the input point cloud. An implicit decoder learns to predict signed distance values given points represented in this feature space.
  arXiv  Detail & Related papers  (2022-07-09T00:14:39Z)
• Scribble-Supervised LiDAR Semantic Segmentation [102.62963605429508]
  We propose using scribbles to annotate LiDAR point clouds and release ScribbleKITTI, the first scribble-annotated dataset for LiDAR semantic segmentation. Our pipeline comprises of three stand-alone contributions that can be combined with any LiDAR semantic segmentation model to achieve up to 95.7% of the fully-supervised performance.
  arXiv  Detail & Related papers  (2022-03-16T11:01:23Z)
• S3CNet: A Sparse Semantic Scene Completion Network for LiDAR Point Clouds [0.16799377888527683]
  We present S3CNet, a sparse convolution based neural network that predicts the semantically completed scene from a single, unified LiDAR point cloud. We show that our proposed method outperforms all counterparts on the 3D task, achieving state-of-the art results on the Semantic KITTI benchmark.
  arXiv  Detail & Related papers  (2020-12-16T20:14:41Z)
• SelfVoxeLO: Self-supervised LiDAR Odometry with Voxel-based Deep Neural Networks [81.64530401885476]
  We propose a self-supervised LiDAR odometry method, dubbed SelfVoxeLO, to tackle these two difficulties. Specifically, we propose a 3D convolution network to process the raw LiDAR data directly, which extracts features that better encode the 3D geometric patterns. We evaluate our method's performances on two large-scale datasets, i.e., KITTI and Apollo-SouthBay.
  arXiv  Detail & Related papers  (2020-10-19T09:23:39Z)
• Cascaded Refinement Network for Point Cloud Completion with Self-supervision [74.80746431691938]
  We introduce a two-branch network for shape completion. The first branch is a cascaded shape completion sub-network to synthesize complete objects. The second branch is an auto-encoder to reconstruct the original partial input.
  arXiv  Detail & Related papers  (2020-10-17T04:56:22Z)
• Scan-based Semantic Segmentation of LiDAR Point Clouds: An Experimental Study [2.6205925938720833]
  State of the art methods use deep neural networks to predict semantic classes for each point in a LiDAR scan. A powerful and efficient way to process LiDAR measurements is to use two-dimensional, image-like projections. We demonstrate various techniques to boost the performance and to improve runtime as well as memory constraints.
  arXiv  Detail & Related papers  (2020-04-06T11:08:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.