P$^2$SDF for Neural Indoor Scene Reconstruction
- URL: http://arxiv.org/abs/2303.00236v1
- Date: Wed, 1 Mar 2023 05:07:48 GMT
- Title: P$^2$SDF for Neural Indoor Scene Reconstruction
- Authors: Jing Li, Jinpeng Yu, Ruoyu Wang, Zhengxin Li, Zhengyu Zhang, Lina Cao,
and Shenghua Gao
- Abstract summary: We propose a novel Pseudo Plane-regularized Signed Distance Field (P$2$SDF) for indoor scene reconstruction.
Experiments show that our P$2$SDF achieves competitive reconstruction performance in Manhattan scenes.
- Score: 29.355255923026597
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Given only a set of images, neural implicit surface representation has shown
its capability in 3D surface reconstruction. However, as the nature of
per-scene optimization is based on the volumetric rendering of color, previous
neural implicit surface reconstruction methods usually fail in low-textured
regions, including the floors, walls, etc., which commonly exist for indoor
scenes. Being aware of the fact that these low-textured regions usually
correspond to planes, without introducing additional ground-truth supervisory
signals or making additional assumptions about the room layout, we propose to
leverage a novel Pseudo Plane-regularized Signed Distance Field (P$^2$SDF) for
indoor scene reconstruction. Specifically, we consider adjacent pixels with
similar colors to be on the same pseudo planes. The plane parameters are then
estimated on the fly during training by an efficient and effective two-step
scheme. Then the signed distances of the points on the planes are regularized
by the estimated plane parameters in the training phase. As the unsupervised
plane segments are usually noisy and inaccurate, we propose to assign different
weights to the sampled points on the plane in plane estimation as well as the
regularization loss. The weights come by fusing the plane segments from
different views. As the sampled rays in the planar regions are redundant,
leading to inefficient training, we further propose a keypoint-guided rays
sampling strategy that attends to the informative textured regions with large
color variations, and the implicit network gets a better reconstruction,
compared with the original uniform ray sampling strategy. Experiments show that
our P$^2$SDF achieves competitive reconstruction performance in Manhattan
scenes. Further, as we do not introduce any additional room layout assumption,
our P$^2$SDF generalizes well to the reconstruction of non-Manhattan scenes.
Related papers
- RaNeuS: Ray-adaptive Neural Surface Reconstruction [87.20343320266215]
We leverage a differentiable radiance field eg NeRF to reconstruct detailed 3D surfaces in addition to producing novel view renderings.
Considering that different methods formulate and optimize the projection from SDF to radiance field with a globally constant Eikonal regularization, we improve with a ray-wise weighting factor.
Our proposed textitRaNeuS are extensively evaluated on both synthetic and real datasets.
arXiv Detail & Related papers (2024-06-14T07:54:25Z) - Indoor Scene Reconstruction with Fine-Grained Details Using Hybrid
Representation and Normal Prior Enhancement [53.10080345190996]
The reconstruction of indoor scenes from multi-view RGB images is challenging due to the coexistence of flat and texture-less regions.
Recent methods leverage neural radiance fields aided by predicted surface normal priors to recover the scene geometry.
This work aims to reconstruct high-fidelity surfaces with fine-grained details by addressing the above limitations.
arXiv Detail & Related papers (2023-09-14T12:05:29Z) - Ground Plane Matters: Picking Up Ground Plane Prior in Monocular 3D
Object Detection [92.75961303269548]
The ground plane prior is a very informative geometry clue in monocular 3D object detection (M3OD)
We propose a Ground Plane Enhanced Network (GPENet) which resolves both issues at one go.
Our GPENet can outperform other methods and achieve state-of-the-art performance, well demonstrating the effectiveness and the superiority of the proposed approach.
arXiv Detail & Related papers (2022-11-03T02:21:35Z) - PlaneDepth: Self-supervised Depth Estimation via Orthogonal Planes [41.517947010531074]
Multiple near frontal-parallel planes based depth estimation demonstrated impressive results in self-supervised monocular depth estimation (MDE)
We propose the PlaneDepth, a novel planes based presentation, including vertical planes and ground planes.
Our method can extract the ground plane in an unsupervised manner, which is important for autonomous driving.
arXiv Detail & Related papers (2022-10-04T13:51:59Z) - Occupancy Planes for Single-view RGB-D Human Reconstruction [120.5818162569105]
Single-view RGB-D human reconstruction with implicit functions is often formulated as per-point classification.
We propose the occupancy planes (OPlanes) representation, which enables to formulate single-view RGB-D human reconstruction as occupancy prediction on planes which slice through the camera's view frustum.
arXiv Detail & Related papers (2022-08-04T17:59:56Z) - Semi-signed neural fitting for surface reconstruction from unoriented
point clouds [53.379712818791894]
We propose SSN-Fitting to reconstruct a better signed distance field.
SSN-Fitting consists of a semi-signed supervision and a loss-based region sampling strategy.
We conduct experiments to demonstrate that SSN-Fitting achieves state-of-the-art performance under different settings.
arXiv Detail & Related papers (2022-06-14T09:40:17Z) - Neural 3D Scene Reconstruction with the Manhattan-world Assumption [58.90559966227361]
This paper addresses the challenge of reconstructing 3D indoor scenes from multi-view images.
Planar constraints can be conveniently integrated into the recent implicit neural representation-based reconstruction methods.
The proposed method outperforms previous methods by a large margin on 3D reconstruction quality.
arXiv Detail & Related papers (2022-05-05T17:59:55Z) - PLNet: Plane and Line Priors for Unsupervised Indoor Depth Estimation [15.751045404065465]
This paper proposes PLNet that leverages the plane and line priors to enhance the depth estimation.
Experiments on NYU Depth V2 and ScanNet show that PLNet outperforms existing methods.
arXiv Detail & Related papers (2021-10-12T09:02:24Z) - Indoor Panorama Planar 3D Reconstruction via Divide and Conquer [36.466149592254965]
Indoor panorama typically consists of human-made structures parallel or perpendicular to gravity.
We leverage this phenomenon to approximate the scene in a 360-degree image with (H)orizontal-planes and (V)ertical-planes.
We create a benchmark for indoor panorama planar reconstruction by extending existing 360 depth datasets with ground truth H&V-planes.
arXiv Detail & Related papers (2021-06-27T07:58:29Z) - PlaneSegNet: Fast and Robust Plane Estimation Using a Single-stage
Instance Segmentation CNN [12.251947429149796]
We propose a real-time deep neural architecture that estimates piece-wise planar regions from a single RGB image.
Our method achieves significantly higher frame-rates and comparable segmentation accuracy against two-stage methods.
arXiv Detail & Related papers (2021-03-29T08:53:05Z) - ManhattanSLAM: Robust Planar Tracking and Mapping Leveraging Mixture of
Manhattan Frames [41.33367060137042]
RGB-D SLAM system is proposed to utilize the structural information in indoor scenes, allowing for accurate tracking and efficient dense mapping on a CPU.
Planar surfels are directly from sparse planes in our map while non-planar surfels are built by extracting superpixels.
We evaluate our method on public benchmarks for pose estimation, drift and reconstruction accuracy, achieving superior performance compared to other state-of-the-art methods.
arXiv Detail & Related papers (2021-03-28T07:11:57Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.