Semantic Segmentation-assisted Scene Completion for LiDAR Point Clouds
- URL: http://arxiv.org/abs/2109.11453v1
- Date: Thu, 23 Sep 2021 15:55:45 GMT
- Title: Semantic Segmentation-assisted Scene Completion for LiDAR Point Clouds
- Authors: Xuemeng Yang, Hao Zou, Xin Kong, Tianxin Huang, Yong Liu, Wanlong Li,
Feng Wen, and Hongbo Zhang
- Abstract summary: We propose an end-to-end semantic segmentation-assisted scene completion network.
The network takes a raw point cloud as input, and merges the features from the segmentation branch into the completion branch hierarchically.
Our method achieves competitive performance on Semantic KITTI dataset with low latency.
- Score: 9.489733900529204
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Outdoor scene completion is a challenging issue in 3D scene understanding,
which plays an important role in intelligent robotics and autonomous driving.
Due to the sparsity of LiDAR acquisition, it is far more complex for 3D scene
completion and semantic segmentation. Since semantic features can provide
constraints and semantic priors for completion tasks, the relationship between
them is worth exploring. Therefore, we propose an end-to-end semantic
segmentation-assisted scene completion network, including a 2D completion
branch and a 3D semantic segmentation branch. Specifically, the network takes a
raw point cloud as input, and merges the features from the segmentation branch
into the completion branch hierarchically to provide semantic information. By
adopting BEV representation and 3D sparse convolution, we can benefit from the
lower operand while maintaining effective expression. Besides, the decoder of
the segmentation branch is used as an auxiliary, which can be discarded in the
inference stage to save computational consumption. Extensive experiments
demonstrate that our method achieves competitive performance on SemanticKITTI
dataset with low latency. Code and models will be released at
https://github.com/jokester-zzz/SSA-SC.
Related papers
- View-Consistent Hierarchical 3D Segmentation Using Ultrametric Feature Fields [52.08335264414515]
We learn a novel feature field within a Neural Radiance Field (NeRF) representing a 3D scene.
Our method takes view-inconsistent multi-granularity 2D segmentations as input and produces a hierarchy of 3D-consistent segmentations as output.
We evaluate our method and several baselines on synthetic datasets with multi-view images and multi-granular segmentation, showcasing improved accuracy and viewpoint-consistency.
arXiv Detail & Related papers (2024-05-30T04:14:58Z) - PointSSC: A Cooperative Vehicle-Infrastructure Point Cloud Benchmark for
Semantic Scene Completion [4.564209472726044]
Semantic Scene Completion aims to jointly generate space occupancies and semantic labels for complex 3D scenes.
PointSSC is the first cooperative vehicle-infrastructure point cloud benchmark for semantic scene completion.
arXiv Detail & Related papers (2023-09-22T08:39:16Z) - A Threefold Review on Deep Semantic Segmentation: Efficiency-oriented,
Temporal and Depth-aware design [77.34726150561087]
We conduct a survey on the most relevant and recent advances in Deep Semantic in the context of vision for autonomous vehicles.
Our main objective is to provide a comprehensive discussion on the main methods, advantages, limitations, results and challenges faced from each perspective.
arXiv Detail & Related papers (2023-03-08T01:29:55Z) - SemSegDepth: A Combined Model for Semantic Segmentation and Depth
Completion [18.19171031755595]
We propose a new end-to-end model for performing semantic segmentation and depth completion jointly.
Our approach relies on RGB and sparse depth as inputs to our model and produces a dense depth map and the corresponding semantic segmentation image.
Experiments done on Virtual KITTI 2 dataset, demonstrate and provide further evidence, that combining both tasks, semantic segmentation and depth completion, in a multi-task network can effectively improve the performance of each task.
arXiv Detail & Related papers (2022-09-01T11:52:11Z) - Dense Supervision Propagation for Weakly Supervised Semantic Segmentation on 3D Point Clouds [59.63231842439687]
We train a semantic point cloud segmentation network with only a small portion of points being labeled.
We propose a cross-sample feature reallocating module to transfer similar features and therefore re-route the gradients across two samples.
Our weakly supervised method with only 10% and 1% of labels can produce compatible results with the fully supervised counterpart.
arXiv Detail & Related papers (2021-07-23T14:34:57Z) - Semantic Scene Completion via Integrating Instances and Scene
in-the-Loop [73.11401855935726]
Semantic Scene Completion aims at reconstructing a complete 3D scene with precise voxel-wise semantics from a single-view depth or RGBD image.
We present Scene-Instance-Scene Network (textitSISNet), which takes advantages of both instance and scene level semantic information.
Our method is capable of inferring fine-grained shape details as well as nearby objects whose semantic categories are easily mixed-up.
arXiv Detail & Related papers (2021-04-08T09:50:30Z) - S3Net: 3D LiDAR Sparse Semantic Segmentation Network [1.330528227599978]
S3Net is a novel convolutional neural network for LiDAR point cloud semantic segmentation.
It adopts an encoder-decoder backbone that consists of Sparse Intra-channel Attention Module (SIntraAM) and Sparse Inter-channel Attention Module (SInterAM)
arXiv Detail & Related papers (2021-03-15T22:15:24Z) - (AF)2-S3Net: Attentive Feature Fusion with Adaptive Feature Selection
for Sparse Semantic Segmentation Network [3.6967381030744515]
We propose AF2-S3Net, an end-to-end encoder-decoder CNN network for 3D LiDAR semantic segmentation.
We present a novel multi-branch attentive feature fusion module in the encoder and a unique adaptive feature selection module with feature map re-weighting in the decoder.
Our experimental results show that the proposed method outperforms the state-of-the-art approaches on the large-scale SemanticKITTI benchmark.
arXiv Detail & Related papers (2021-02-08T21:04:21Z) - S3CNet: A Sparse Semantic Scene Completion Network for LiDAR Point
Clouds [0.16799377888527683]
We present S3CNet, a sparse convolution based neural network that predicts the semantically completed scene from a single, unified LiDAR point cloud.
We show that our proposed method outperforms all counterparts on the 3D task, achieving state-of-the art results on the Semantic KITTI benchmark.
arXiv Detail & Related papers (2020-12-16T20:14:41Z) - Improving Point Cloud Semantic Segmentation by Learning 3D Object
Detection [102.62963605429508]
Point cloud semantic segmentation plays an essential role in autonomous driving.
Current 3D semantic segmentation networks focus on convolutional architectures that perform great for well represented classes.
We propose a novel Aware 3D Semantic Detection (DASS) framework that explicitly leverages localization features from an auxiliary 3D object detection task.
arXiv Detail & Related papers (2020-09-22T14:17:40Z) - Depth Based Semantic Scene Completion with Position Importance Aware
Loss [52.06051681324545]
PALNet is a novel hybrid network for semantic scene completion.
It extracts both 2D and 3D features from multi-stages using fine-grained depth information.
It is beneficial for recovering key details like the boundaries of objects and the corners of the scene.
arXiv Detail & Related papers (2020-01-29T07:05:52Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.