RfD-Net: Point Scene Understanding by Semantic Instance Reconstruction
- URL: http://arxiv.org/abs/2011.14744v1
- Date: Mon, 30 Nov 2020 12:58:05 GMT
- Title: RfD-Net: Point Scene Understanding by Semantic Instance Reconstruction
- Authors: Yinyu Nie, Ji Hou, Xiaoguang Han, Matthias Nie{\ss}ner
- Abstract summary: We introduce RfD-Net that jointly detects and reconstructs dense object surfaces directly from point clouds.
We decouple the instance reconstruction into global object localization and local shape prediction.
Our approach consistently outperforms the state-of-the-arts and improves over 11 of mesh IoU in object reconstruction.
- Score: 19.535169371240073
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Semantic scene understanding from point clouds is particularly challenging as
the points reflect only a sparse set of the underlying 3D geometry. Previous
works often convert point cloud into regular grids (e.g. voxels or bird-eye
view images), and resort to grid-based convolutions for scene understanding. In
this work, we introduce RfD-Net that jointly detects and reconstructs dense
object surfaces directly from raw point clouds. Instead of representing scenes
with regular grids, our method leverages the sparsity of point cloud data and
focuses on predicting shapes that are recognized with high objectness. With
this design, we decouple the instance reconstruction into global object
localization and local shape prediction. It not only eases the difficulty of
learning 2-D manifold surfaces from sparse 3D space, the point clouds in each
object proposal convey shape details that support implicit function learning to
reconstruct any high-resolution surfaces. Our experiments indicate that
instance detection and reconstruction present complementary effects, where the
shape prediction head shows consistent effects on improving object detection
with modern 3D proposal network backbones. The qualitative and quantitative
evaluations further demonstrate that our approach consistently outperforms the
state-of-the-arts and improves over 11 of mesh IoU in object reconstruction.
Related papers
- Self-supervised 3D Point Cloud Completion via Multi-view Adversarial Learning [61.14132533712537]
We propose MAL-SPC, a framework that effectively leverages both object-level and category-specific geometric similarities to complete missing structures.
Our MAL-SPC does not require any 3D complete supervision and only necessitates a single partial point cloud for each object.
arXiv Detail & Related papers (2024-07-13T06:53:39Z) - Flattening-Net: Deep Regular 2D Representation for 3D Point Cloud
Analysis [66.49788145564004]
We present an unsupervised deep neural architecture called Flattening-Net to represent irregular 3D point clouds of arbitrary geometry and topology.
Our methods perform favorably against the current state-of-the-art competitors.
arXiv Detail & Related papers (2022-12-17T15:05:25Z) - Single-view 3D Mesh Reconstruction for Seen and Unseen Categories [69.29406107513621]
Single-view 3D Mesh Reconstruction is a fundamental computer vision task that aims at recovering 3D shapes from single-view RGB images.
This paper tackles Single-view 3D Mesh Reconstruction, to study the model generalization on unseen categories.
We propose an end-to-end two-stage network, GenMesh, to break the category boundaries in reconstruction.
arXiv Detail & Related papers (2022-08-04T14:13:35Z) - Point Scene Understanding via Disentangled Instance Mesh Reconstruction [21.92736190195887]
We propose aDisentangled Instance Mesh Reconstruction (DIMR) framework for effective point scene understanding.
A segmentation-based backbone is applied to reduce false positive object proposals.
We leverage a mesh-aware latent code space to disentangle the processes of shape completion and mesh generation.
arXiv Detail & Related papers (2022-03-31T06:36:07Z) - Learnable Triangulation for Deep Learning-based 3D Reconstruction of
Objects of Arbitrary Topology from Single RGB Images [12.693545159861857]
We propose a novel deep reinforcement learning-based approach for 3D object reconstruction from monocular images.
The proposed method outperforms the state-of-the-art in terms of visual quality, reconstruction accuracy, and computational time.
arXiv Detail & Related papers (2021-09-24T09:44:22Z) - From Points to Multi-Object 3D Reconstruction [71.17445805257196]
We propose a method to detect and reconstruct multiple 3D objects from a single RGB image.
A keypoint detector localizes objects as center points and directly predicts all object properties, including 9-DoF bounding boxes and 3D shapes.
The presented approach performs lightweight reconstruction in a single-stage, it is real-time capable, fully differentiable and end-to-end trainable.
arXiv Detail & Related papers (2020-12-21T18:52:21Z) - 3D Object Classification on Partial Point Clouds: A Practical
Perspective [91.81377258830703]
A point cloud is a popular shape representation adopted in 3D object classification.
This paper introduces a practical setting to classify partial point clouds of object instances under any poses.
A novel algorithm in an alignment-classification manner is proposed in this paper.
arXiv Detail & Related papers (2020-12-18T04:00:56Z) - Learning Occupancy Function from Point Clouds for Surface Reconstruction [6.85316573653194]
Implicit function based surface reconstruction has been studied for a long time to recover 3D shapes from point clouds sampled from surfaces.
This paper proposes a novel method for learning occupancy functions from sparse point clouds and achieves better performance on challenging surface reconstruction tasks.
arXiv Detail & Related papers (2020-10-22T02:07:29Z) - Pix2Surf: Learning Parametric 3D Surface Models of Objects from Images [64.53227129573293]
We investigate the problem of learning to generate 3D parametric surface representations for novel object instances, as seen from one or more views.
We design neural networks capable of generating high-quality parametric 3D surfaces which are consistent between views.
Our method is supervised and trained on a public dataset of shapes from common object categories.
arXiv Detail & Related papers (2020-08-18T06:33:40Z) - Meshing Point Clouds with Predicted Intrinsic-Extrinsic Ratio Guidance [30.863194319818223]
We propose to leverage the input point cloud as much as possible, by only adding connectivity information to existing points.
Our key innovation is a surrogate of local connectivity, calculated by comparing the intrinsic/extrinsic metrics.
We demonstrate that our method can not only preserve details, handle ambiguous structures, but also possess strong generalizability to unseen categories.
arXiv Detail & Related papers (2020-07-17T22:36:00Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.