Learning to Segment 3D Point Clouds in 2D Image Space
- URL: http://arxiv.org/abs/2003.05593v4
- Date: Wed, 7 Oct 2020 23:27:31 GMT
- Title: Learning to Segment 3D Point Clouds in 2D Image Space
- Authors: Yecheng Lyu and Xinming Huang and Ziming Zhang
- Abstract summary: We show how to efficiently project 3D point clouds into a 2D image space.
Traditional 2D convolutional neural networks (CNNs) such as U-Net can be applied for segmentation.
- Score: 20.119802932358333
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In contrast to the literature where local patterns in 3D point clouds are
captured by customized convolutional operators, in this paper we study the
problem of how to effectively and efficiently project such point clouds into a
2D image space so that traditional 2D convolutional neural networks (CNNs) such
as U-Net can be applied for segmentation. To this end, we are motivated by
graph drawing and reformulate it as an integer programming problem to learn the
topology-preserving graph-to-grid mapping for each individual point cloud. To
accelerate the computation in practice, we further propose a novel hierarchical
approximate algorithm. With the help of the Delaunay triangulation for graph
construction from point clouds and a multi-scale U-Net for segmentation, we
manage to demonstrate the state-of-the-art performance on ShapeNet and PartNet,
respectively, with significant improvement over the literature. Code is
available at https://github.com/Zhang-VISLab.
Related papers
- Clustering based Point Cloud Representation Learning for 3D Analysis [80.88995099442374]
We propose a clustering based supervised learning scheme for point cloud analysis.
Unlike current de-facto, scene-wise training paradigm, our algorithm conducts within-class clustering on the point embedding space.
Our algorithm shows notable improvements on famous point cloud segmentation datasets.
arXiv Detail & Related papers (2023-07-27T03:42:12Z) - Flattening-Net: Deep Regular 2D Representation for 3D Point Cloud
Analysis [66.49788145564004]
We present an unsupervised deep neural architecture called Flattening-Net to represent irregular 3D point clouds of arbitrary geometry and topology.
Our methods perform favorably against the current state-of-the-art competitors.
arXiv Detail & Related papers (2022-12-17T15:05:25Z) - GraphCSPN: Geometry-Aware Depth Completion via Dynamic GCNs [49.55919802779889]
We propose a Graph Convolution based Spatial Propagation Network (GraphCSPN) as a general approach for depth completion.
In this work, we leverage convolution neural networks as well as graph neural networks in a complementary way for geometric representation learning.
Our method achieves the state-of-the-art performance, especially when compared in the case of using only a few propagation steps.
arXiv Detail & Related papers (2022-10-19T17:56:03Z) - Voint Cloud: Multi-View Point Cloud Representation for 3D Understanding [80.04281842702294]
We introduce the concept of the multi-view point cloud (Voint cloud) representing each 3D point as a set of features extracted from several view-points.
This novel 3D Voint cloud representation combines the compactness of 3D point cloud representation with the natural view-awareness of multi-view representation.
We deploy a Voint neural network (VointNet) with a theoretically established functional form to learn representations in the Voint space.
arXiv Detail & Related papers (2021-11-30T13:08:19Z) - TreeGCN-ED: Encoding Point Cloud using a Tree-Structured Graph Network [24.299931323012757]
This work proposes an autoencoder based framework to generate robust embeddings for point clouds.
We demonstrate the applicability of the proposed framework in applications like: 3D point cloud completion and Single image based 3D reconstruction.
arXiv Detail & Related papers (2021-10-07T03:52:56Z) - LatticeNet: Fast Spatio-Temporal Point Cloud Segmentation Using
Permutohedral Lattices [27.048998326468688]
Deep convolutional neural networks (CNNs) have shown outstanding performance in the task of semantically segmenting images.
Here, we propose LatticeNet, a novel approach for 3D semantic segmentation, which takes raw point clouds as input.
We present results of 3D segmentation on multiple datasets where our method achieves state-of-the-art performance.
arXiv Detail & Related papers (2021-08-09T10:17:27Z) - Learning point embedding for 3D data processing [2.12121796606941]
Current point-based methods are essentially spatial relationship processing networks.
Our architecture, PE-Net, learns the representation of point clouds in high-dimensional space.
Experiments show that PE-Net achieves the state-of-the-art performance in multiple challenging datasets.
arXiv Detail & Related papers (2021-07-19T00:25:28Z) - Exploiting Local Geometry for Feature and Graph Construction for Better
3D Point Cloud Processing with Graph Neural Networks [22.936590869919865]
We propose improvements in point representations and local neighborhood graph construction within the general framework of graph neural networks.
We show that the proposed network achieves faster training convergence, i.e. 40% less epochs for classification.
arXiv Detail & Related papers (2021-03-28T21:34:59Z) - ParaNet: Deep Regular Representation for 3D Point Clouds [62.81379889095186]
ParaNet is a novel end-to-end deep learning framework for representing 3D point clouds.
It converts an irregular 3D point cloud into a regular 2D color image, named point geometry image (PGI)
In contrast to conventional regular representation modalities based on multi-view projection and voxelization, the proposed representation is differentiable and reversible.
arXiv Detail & Related papers (2020-12-05T13:19:55Z) - GRNet: Gridding Residual Network for Dense Point Cloud Completion [54.43648460932248]
Estimating the complete 3D point cloud from an incomplete one is a key problem in many vision and robotics applications.
We propose a novel Gridding Residual Network (GRNet) for point cloud completion.
Experimental results indicate that the proposed GRNet performs favorably against state-of-the-art methods on the ShapeNet, Completion3D, and KITTI benchmarks.
arXiv Detail & Related papers (2020-06-06T02:46:39Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.