3D Reconstruction and New View Synthesis of Indoor Environments based on a Dual Neural Radiance Field

• URL: http://arxiv.org/abs/2401.14726v2
• Date: Fri, 19 Jul 2024 09:35:18 GMT
• Title: 3D Reconstruction and New View Synthesis of Indoor Environments based on a Dual Neural Radiance Field
• Authors: Zhenyu Bao, Guibiao Liao, Zhongyuan Zhao, Kanglin Liu, Qing Li, Guoping Qiu,
• Abstract summary: We develop a dual neural radiance field (Du-NeRF) to simultaneously achieve high-quality geometry reconstruction and view rendering. One of the innovative features of Du-NeRF is that it decouples a view-independent component from the density field and uses it as a label to supervise the learning process of the SDF field.
• Score: 17.709306549110153
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Simultaneously achieving 3D reconstruction and new view synthesis for indoor environments has widespread applications but is technically very challenging. State-of-the-art methods based on implicit neural functions can achieve excellent 3D reconstruction results, but their performances on new view synthesis can be unsatisfactory. The exciting development of neural radiance field (NeRF) has revolutionized new view synthesis, however, NeRF-based models can fail to reconstruct clean geometric surfaces. We have developed a dual neural radiance field (Du-NeRF) to simultaneously achieve high-quality geometry reconstruction and view rendering. Du-NeRF contains two geometric fields, one derived from the SDF field to facilitate geometric reconstruction and the other derived from the density field to boost new view synthesis. One of the innovative features of Du-NeRF is that it decouples a view-independent component from the density field and uses it as a label to supervise the learning process of the SDF field. This reduces shape-radiance ambiguity and enables geometry and color to benefit from each other during the learning process. Extensive experiments demonstrate that Du-NeRF can significantly improve the performance of novel view synthesis and 3D reconstruction for indoor environments and it is particularly effective in constructing areas containing fine geometries that do not obey multi-view color consistency.

Related papers

• AniSDF: Fused-Granularity Neural Surfaces with Anisotropic Encoding for High-Fidelity 3D Reconstruction [55.69271635843385]
  We present AniSDF, a novel approach that learns fused-granularity neural surfaces with physics-based encoding for high-fidelity 3D reconstruction. Our method boosts the quality of SDF-based methods by a great scale in both geometry reconstruction and novel-view synthesis.
  arXiv  Detail & Related papers  (2024-10-02T03:10:38Z)
• RaNeuS: Ray-adaptive Neural Surface Reconstruction [87.20343320266215]
  We leverage a differentiable radiance field eg NeRF to reconstruct detailed 3D surfaces in addition to producing novel view renderings. Considering that different methods formulate and optimize the projection from SDF to radiance field with a globally constant Eikonal regularization, we improve with a ray-wise weighting factor. Our proposed textitRaNeuS are extensively evaluated on both synthetic and real datasets.
  arXiv  Detail & Related papers  (2024-06-14T07:54:25Z)
• High-fidelity 3D Reconstruction of Plants using Neural Radiance Field [10.245620447865456]
  We present a novel plant dataset comprising real plant images from production environments. This dataset is a first-of-its-kind initiative aimed at comprehensively exploring the advantages and limitations of NeRF in agricultural contexts.
  arXiv  Detail & Related papers  (2023-11-07T17:31:27Z)
• BeyondPixels: A Comprehensive Review of the Evolution of Neural Radiance Fields [1.1531932979578041]
  NeRF, short for Neural Radiance Fields, is a recent innovation that uses AI algorithms to create 3D objects from 2D images. This survey reviews recent advances in NeRF and categorizes them according to their architectural designs.
  arXiv  Detail & Related papers  (2023-06-05T16:10:21Z)
• Neural Radiance Fields (NeRFs): A Review and Some Recent Developments [0.0]
  Neural Radiance Field (NeRF) is a framework that represents a 3D scene in the weights of a fully connected neural network. NeRFs have become a popular field of research as recent developments have been made that expand the performance and capabilities of the base framework.
  arXiv  Detail & Related papers  (2023-04-30T03:23:58Z)
• DehazeNeRF: Multiple Image Haze Removal and 3D Shape Reconstruction using Neural Radiance Fields [56.30120727729177]
  We introduce DehazeNeRF as a framework that robustly operates in hazy conditions. We demonstrate successful multi-view haze removal, novel view synthesis, and 3D shape reconstruction where existing approaches fail.
  arXiv  Detail & Related papers  (2023-03-20T18:03:32Z)
• NeRFMeshing: Distilling Neural Radiance Fields into Geometrically-Accurate 3D Meshes [56.31855837632735]
  We propose a compact and flexible architecture that enables easy 3D surface reconstruction from any NeRF-driven approach. Our final 3D mesh is physically accurate and can be rendered in real time on an array of devices.
  arXiv  Detail & Related papers  (2023-03-16T16:06:03Z)
• NeuralUDF: Learning Unsigned Distance Fields for Multi-view Reconstruction of Surfaces with Arbitrary Topologies [87.06532943371575]
  We present a novel method, called NeuralUDF, for reconstructing surfaces with arbitrary topologies from 2D images via volume rendering. In this paper, we propose to represent surfaces as the Unsigned Distance Function (UDF) and develop a new volume rendering scheme to learn the neural UDF representation.
  arXiv  Detail & Related papers  (2022-11-25T15:21:45Z)
• UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction [61.17219252031391]
  We present a novel method for reconstructing surfaces from multi-view images using Neural implicit 3D representations. Our key insight is that implicit surface models and radiance fields can be formulated in a unified way, enabling both surface and volume rendering. Our experiments demonstrate that we outperform NeRF in terms of reconstruction quality while performing on par with IDR without requiring masks.
  arXiv  Detail & Related papers  (2021-04-20T15:59:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.