PlaNeRF: SVD Unsupervised 3D Plane Regularization for NeRF Large-Scale Scene Reconstruction

• URL: http://arxiv.org/abs/2305.16914v4
• Date: Sun, 5 Nov 2023 09:30:46 GMT
• Title: PlaNeRF: SVD Unsupervised 3D Plane Regularization for NeRF Large-Scale Scene Reconstruction
• Authors: Fusang Wang, Arnaud Louys, Nathan Piasco, Moussab Bennehar, Luis Rold\~ao, Dzmitry Tsishkou
• Abstract summary: Neural Radiance Fields (NeRF) enable 3D scene reconstruction from 2D images and camera poses for Novel View Synthesis (NVS) NeRF often suffers from overfitting to training views, leading to poor geometry reconstruction. We propose a new method to improve NeRF's 3D structure using only RGB images and semantic maps.
• Score: 2.2369578015657954
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Neural Radiance Fields (NeRF) enable 3D scene reconstruction from 2D images and camera poses for Novel View Synthesis (NVS). Although NeRF can produce photorealistic results, it often suffers from overfitting to training views, leading to poor geometry reconstruction, especially in low-texture areas. This limitation restricts many important applications which require accurate geometry, such as extrapolated NVS, HD mapping and scene editing. To address this limitation, we propose a new method to improve NeRF's 3D structure using only RGB images and semantic maps. Our approach introduces a novel plane regularization based on Singular Value Decomposition (SVD), that does not rely on any geometric prior. In addition, we leverage the Structural Similarity Index Measure (SSIM) in our loss design to properly initialize the volumetric representation of NeRF. Quantitative and qualitative results show that our method outperforms popular regularization approaches in accurate geometry reconstruction for large-scale outdoor scenes and achieves SoTA rendering quality on the KITTI-360 NVS benchmark.

Related papers

• Towards Degradation-Robust Reconstruction in Generalizable NeRF [58.33351079982745]
  Generalizable Radiance Field (GNeRF) across scenes has been proven to be an effective way to avoid per-scene optimization. There has been limited research on the robustness of GNeRFs to different types of degradation present in the source images.
  arXiv  Detail & Related papers  (2024-11-18T16:13:47Z)
• PGSR: Planar-based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstruction [37.14913599050765]
  We propose a fast planar-based Gaussian splatting reconstruction representation (PGSR) to achieve high-fidelity surface reconstruction. We then introduce single-view geometric, multi-view photometric, and geometric regularization to preserve global geometric accuracy. Our method achieves fast training and rendering while maintaining high-fidelity rendering and geometric reconstruction, outperforming 3DGS-based and NeRF-based methods.
  arXiv  Detail & Related papers  (2024-06-10T17:59:01Z)
• GTR: Improving Large 3D Reconstruction Models through Geometry and Texture Refinement [51.97726804507328]
  We propose a novel approach for 3D mesh reconstruction from multi-view images. Our method takes inspiration from large reconstruction models that use a transformer-based triplane generator and a Neural Radiance Field (NeRF) model trained on multi-view images.
  arXiv  Detail & Related papers  (2024-06-09T05:19:24Z)
• ReconFusion: 3D Reconstruction with Diffusion Priors [104.73604630145847]
  We present ReconFusion to reconstruct real-world scenes using only a few photos. Our approach leverages a diffusion prior for novel view synthesis, trained on synthetic and multiview datasets. Our method synthesizes realistic geometry and texture in underconstrained regions while preserving the appearance of observed regions.
  arXiv  Detail & Related papers  (2023-12-05T18:59:58Z)
• Improving Neural Radiance Fields with Depth-aware Optimization for Novel View Synthesis [12.3338393483795]
  We propose SfMNeRF, a method to better synthesize novel views as well as reconstruct the 3D-scene geometry. SfMNeRF employs the epipolar, photometric consistency, depth smoothness, and position-of-matches constraints to explicitly reconstruct the 3D-scene structure. Experiments on two public datasets demonstrate that SfMNeRF surpasses state-of-the-art approaches.
  arXiv  Detail & Related papers  (2023-04-11T13:37:17Z)
• Clean-NeRF: Reformulating NeRF to account for View-Dependent Observations [67.54358911994967]
  This paper proposes Clean-NeRF for accurate 3D reconstruction and novel view rendering in complex scenes. Clean-NeRF can be implemented as a plug-in that can immediately benefit existing NeRF-based methods without additional input.
  arXiv  Detail & Related papers  (2023-03-26T12:24:31Z)
• Learning Neural Radiance Fields from Multi-View Geometry [1.1011268090482573]
  We present a framework, called MVG-NeRF, that combines Multi-View Geometry algorithms and Neural Radiance Fields (NeRF) for image-based 3D reconstruction. NeRF has revolutionized the field of implicit 3D representations, mainly due to a differentiable rendering formulation that enables high-quality and geometry-aware novel view synthesis.
  arXiv  Detail & Related papers  (2022-10-24T08:53:35Z)
• NeuRIS: Neural Reconstruction of Indoor Scenes Using Normal Priors [84.66706400428303]
  We propose a new method, named NeuRIS, for high quality reconstruction of indoor scenes. NeuRIS integrates estimated normal of indoor scenes as a prior in a neural rendering framework. Experiments show that NeuRIS significantly outperforms the state-of-the-art methods in terms of reconstruction quality.
  arXiv  Detail & Related papers  (2022-06-27T19:22:03Z)
• NeRFusion: Fusing Radiance Fields for Large-Scale Scene Reconstruction [50.54946139497575]
  We propose NeRFusion, a method that combines the advantages of NeRF and TSDF-based fusion techniques to achieve efficient large-scale reconstruction and photo-realistic rendering. We demonstrate that NeRFusion achieves state-of-the-art quality on both large-scale indoor and small-scale object scenes, with substantially faster reconstruction than NeRF and other recent methods.
  arXiv  Detail & Related papers  (2022-03-21T18:56:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.