Normal-NeRF: Ambiguity-Robust Normal Estimation for Highly Reflective Scenes

• URL: http://arxiv.org/abs/2501.09460v1
• Date: Thu, 16 Jan 2025 10:42:29 GMT
• Title: Normal-NeRF: Ambiguity-Robust Normal Estimation for Highly Reflective Scenes
• Authors: Ji Shi, Xianghua Ying, Ruohao Guo, Bowei Xing, Wenzhen Yue,
• Abstract summary: We introduce a transmittance-gradient-based normal estimation technique that remains robust even under ambiguous shape conditions. Our proposed method achieves robust reconstruction and high-fidelity rendering of scenes featuring both highly specular reflections and intricate geometric structures.
• Score: 11.515561389102174
• License:
• Abstract: Neural Radiance Fields (NeRF) often struggle with reconstructing and rendering highly reflective scenes. Recent advancements have developed various reflection-aware appearance models to enhance NeRF's capability to render specular reflections. However, the robust reconstruction of highly reflective scenes is still hindered by the inherent shape ambiguity on specular surfaces. Existing methods typically rely on additional geometry priors to regularize the shape prediction, but this can lead to oversmoothed geometry in complex scenes. Observing the critical role of surface normals in parameterizing reflections, we introduce a transmittance-gradient-based normal estimation technique that remains robust even under ambiguous shape conditions. Furthermore, we propose a dual activated densities module that effectively bridges the gap between smooth surface normals and sharp object boundaries. Combined with a reflection-aware appearance model, our proposed method achieves robust reconstruction and high-fidelity rendering of scenes featuring both highly specular reflections and intricate geometric structures. Extensive experiments demonstrate that our method outperforms existing state-of-the-art methods on various datasets.

Related papers

• Acquisition of Spatially-Varying Reflectance and Surface Normals via Polarized Reflectance Fields [15.653977591138682]
  Accurately measuring the geometry and spatially-varying reflectance of real-world objects is a complex task. We propose a novel approach using polarized reflectance field capture and a comprehensive statistical analysis algorithm. We showcase the captured shapes and reflectance of diverse objects with a wide material range, spanning from highly diffuse to highly glossy.
  arXiv  Detail & Related papers  (2024-12-13T00:39:55Z)
• Planar Reflection-Aware Neural Radiance Fields [32.709468082010126]
  We introduce a reflection-aware NeRF that jointly models planar reflectors, such as windows, and explicitly casts reflected rays to capture the source of the high-frequency reflections. Rendering along the primary ray results in a clean, reflection-free view, while explicitly rendering along the reflected ray allows us to reconstruct highly detailed reflections.
  arXiv  Detail & Related papers  (2024-11-07T18:55:08Z)
• AniSDF: Fused-Granularity Neural Surfaces with Anisotropic Encoding for High-Fidelity 3D Reconstruction [55.69271635843385]
  We present AniSDF, a novel approach that learns fused-granularity neural surfaces with physics-based encoding for high-fidelity 3D reconstruction. Our method boosts the quality of SDF-based methods by a great scale in both geometry reconstruction and novel-view synthesis.
  arXiv  Detail & Related papers  (2024-10-02T03:10:38Z)
• Rethinking Directional Parameterization in Neural Implicit Surface Reconstruction [7.970528879271508]
  Multi-view 3D surface reconstruction using neural implicit representations has made notable progress. However, their effectiveness in reconstructing objects with specular or complex surfaces is typically biased by the directional parameterization used in their view-dependent radiance network. We propose a novel hybrid directional parameterization to address their limitations in a unified form.
  arXiv  Detail & Related papers  (2024-09-11T00:44:31Z)
• ND-SDF: Learning Normal Deflection Fields for High-Fidelity Indoor Reconstruction [50.07671826433922]
  It is non-trivial to simultaneously recover meticulous geometry and preserve smoothness across regions with differing characteristics. We propose ND-SDF, which learns a Normal Deflection field to represent the angular deviation between the scene normal and the prior normal. Our method not only obtains smooth weakly textured regions such as walls and floors but also preserves the geometric details of complex structures.
  arXiv  Detail & Related papers  (2024-08-22T17:59:01Z)
• NeRSP: Neural 3D Reconstruction for Reflective Objects with Sparse Polarized Images [62.752710734332894]
  NeRSP is a Neural 3D reconstruction technique for Reflective surfaces with Sparse Polarized images. We derive photometric and geometric cues from the polarimetric image formation model and multiview azimuth consistency. We achieve the state-of-the-art surface reconstruction results with only 6 views as input.
  arXiv  Detail & Related papers  (2024-06-11T09:53:18Z)
• REFRAME: Reflective Surface Real-Time Rendering for Mobile Devices [51.983541908241726]
  This work tackles the challenging task of achieving real-time novel view synthesis for reflective surfaces across various scenes. Existing real-time rendering methods, especially those based on meshes, often have subpar performance in modeling surfaces with rich view-dependent appearances. We decompose the color into diffuse and specular, and model the specular color in the reflected direction based on a neural environment map.
  arXiv  Detail & Related papers  (2024-03-25T07:07:50Z)
• UniSDF: Unifying Neural Representations for High-Fidelity 3D Reconstruction of Complex Scenes with Reflections [87.191742674543]
  We propose UniSDF, a general purpose 3D reconstruction method that can reconstruct large complex scenes with reflections. Our method is able to robustly reconstruct complex large-scale scenes with fine details and reflective surfaces, leading to the best overall performance.
  arXiv  Detail & Related papers  (2023-12-20T18:59:42Z)
• TraM-NeRF: Tracing Mirror and Near-Perfect Specular Reflections through Neural Radiance Fields [3.061835990893184]
  Implicit representations like Neural Radiance Fields (NeRF) showed impressive results for rendering of complex scenes with fine details. We present a novel reflection tracing method tailored for the involved volume rendering within NeRF. We derive efficient strategies for importance sampling and the transmittance computation along rays from only few samples.
  arXiv  Detail & Related papers  (2023-10-16T17:59:56Z)
• Ref-NeuS: Ambiguity-Reduced Neural Implicit Surface Learning for Multi-View Reconstruction with Reflection [24.23826907954389]
  Ref-NeuS aims to reduce ambiguity by attenuating the effect of reflective surfaces. We show that our model achieves high-quality surface reconstruction on reflective surfaces and outperforms the state-of-the-arts by a large margin.
  arXiv  Detail & Related papers  (2023-03-20T03:08:22Z)
• Towards High Fidelity Monocular Face Reconstruction with Rich Reflectance using Self-supervised Learning and Ray Tracing [49.759478460828504]
  Methods combining deep neural network encoders with differentiable rendering have opened up the path for very fast monocular reconstruction of geometry, lighting and reflectance. ray tracing was introduced for monocular face reconstruction within a classic optimization-based framework. We propose a new method that greatly improves reconstruction quality and robustness in general scenes.
  arXiv  Detail & Related papers  (2021-03-29T08:58:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.