Point Scene Understanding via Disentangled Instance Mesh Reconstruction
- URL: http://arxiv.org/abs/2203.16832v1
- Date: Thu, 31 Mar 2022 06:36:07 GMT
- Title: Point Scene Understanding via Disentangled Instance Mesh Reconstruction
- Authors: Jiaxiang Tang, Xiaokang Chen, Jingbo Wang, Gang Zeng
- Abstract summary: We propose aDisentangled Instance Mesh Reconstruction (DIMR) framework for effective point scene understanding.
A segmentation-based backbone is applied to reduce false positive object proposals.
We leverage a mesh-aware latent code space to disentangle the processes of shape completion and mesh generation.
- Score: 21.92736190195887
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Semantic scene reconstruction from point cloud is an essential and
challenging task for 3D scene understanding. This task requires not only to
recognize each instance in the scene, but also to recover their geometries
based on the partial observed point cloud. Existing methods usually attempt to
directly predict occupancy values of the complete object based on incomplete
point cloud proposals from a detection-based backbone. However, this framework
always fails to reconstruct high fidelity mesh due to the obstruction of
various detected false positive object proposals and the ambiguity of
incomplete point observations for learning occupancy values of complete
objects. To circumvent the hurdle, we propose a Disentangled Instance Mesh
Reconstruction (DIMR) framework for effective point scene understanding. A
segmentation-based backbone is applied to reduce false positive object
proposals, which further benefits our exploration on the relationship between
recognition and reconstruction. Based on the accurate proposals, we leverage a
mesh-aware latent code space to disentangle the processes of shape completion
and mesh generation, relieving the ambiguity caused by the incomplete point
observations. Furthermore, with access to the CAD model pool at test time, our
model can also be used to improve the reconstruction quality by performing mesh
retrieval without extra training. We thoroughly evaluate the reconstructed mesh
quality with multiple metrics, and demonstrate the superiority of our method on
the challenging ScanNet dataset.
Related papers
- Self-supervised 3D Point Cloud Completion via Multi-view Adversarial Learning [61.14132533712537]
We propose MAL-SPC, a framework that effectively leverages both object-level and category-specific geometric similarities to complete missing structures.
Our MAL-SPC does not require any 3D complete supervision and only necessitates a single partial point cloud for each object.
arXiv Detail & Related papers (2024-07-13T06:53:39Z) - Shape Anchor Guided Holistic Indoor Scene Understanding [9.463220988312218]
We propose a shape anchor guided learning strategy (AncLearn) for robust holistic indoor scene understanding.
AncLearn generates anchors that dynamically fit instance surfaces to (i) unmix noise and target-related features for offering reliable proposals at the detection stage.
We embed AncLearn into a reconstruction-from-detection learning system (AncRec) to generate high-quality semantic scene models.
arXiv Detail & Related papers (2023-09-20T08:30:20Z) - Semi-signed neural fitting for surface reconstruction from unoriented
point clouds [53.379712818791894]
We propose SSN-Fitting to reconstruct a better signed distance field.
SSN-Fitting consists of a semi-signed supervision and a loss-based region sampling strategy.
We conduct experiments to demonstrate that SSN-Fitting achieves state-of-the-art performance under different settings.
arXiv Detail & Related papers (2022-06-14T09:40:17Z) - High-resolution Iterative Feedback Network for Camouflaged Object
Detection [128.893782016078]
Spotting camouflaged objects that are visually assimilated into the background is tricky for object detection algorithms.
We aim to extract the high-resolution texture details to avoid the detail degradation that causes blurred vision in edges and boundaries.
We introduce a novel HitNet to refine the low-resolution representations by high-resolution features in an iterative feedback manner.
arXiv Detail & Related papers (2022-03-22T11:20:21Z) - Locally Aware Piecewise Transformation Fields for 3D Human Mesh
Registration [67.69257782645789]
We propose piecewise transformation fields that learn 3D translation vectors to map any query point in posed space to its correspond position in rest-pose space.
We show that fitting parametric models with poses by our network results in much better registration quality, especially for extreme poses.
arXiv Detail & Related papers (2021-04-16T15:16:09Z) - Unsupervised Learning of 3D Object Categories from Videos in the Wild [75.09720013151247]
We focus on learning a model from multiple views of a large collection of object instances.
We propose a new neural network design, called warp-conditioned ray embedding (WCR), which significantly improves reconstruction.
Our evaluation demonstrates performance improvements over several deep monocular reconstruction baselines on existing benchmarks.
arXiv Detail & Related papers (2021-03-30T17:57:01Z) - RfD-Net: Point Scene Understanding by Semantic Instance Reconstruction [19.535169371240073]
We introduce RfD-Net that jointly detects and reconstructs dense object surfaces directly from point clouds.
We decouple the instance reconstruction into global object localization and local shape prediction.
Our approach consistently outperforms the state-of-the-arts and improves over 11 of mesh IoU in object reconstruction.
arXiv Detail & Related papers (2020-11-30T12:58:05Z) - Skeleton-bridged Point Completion: From Global Inference to Local
Adjustment [48.2757171993437]
We propose a skeleton-bridged point completion network (SK-PCN) for shape completion.
Given a partial scan, our method first predicts its 3D skeleton to obtain the global structure.
We decouple the shape completion into structure estimation and surface reconstruction, which eases the learning difficulty.
arXiv Detail & Related papers (2020-10-14T22:49:30Z) - Meshing Point Clouds with Predicted Intrinsic-Extrinsic Ratio Guidance [30.863194319818223]
We propose to leverage the input point cloud as much as possible, by only adding connectivity information to existing points.
Our key innovation is a surrogate of local connectivity, calculated by comparing the intrinsic/extrinsic metrics.
We demonstrate that our method can not only preserve details, handle ambiguous structures, but also possess strong generalizability to unseen categories.
arXiv Detail & Related papers (2020-07-17T22:36:00Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.