Ref-NeuS: Ambiguity-Reduced Neural Implicit Surface Learning for Multi-View Reconstruction with Reflection

• URL: http://arxiv.org/abs/2303.10840v2
• Date: Mon, 17 Jul 2023 07:24:30 GMT
• Title: Ref-NeuS: Ambiguity-Reduced Neural Implicit Surface Learning for Multi-View Reconstruction with Reflection
• Authors: Wenhang Ge and Tao Hu and Haoyu Zhao and Shu Liu and Ying-Cong Chen
• Abstract summary: Ref-NeuS aims to reduce ambiguity by attenuating the effect of reflective surfaces. We show that our model achieves high-quality surface reconstruction on reflective surfaces and outperforms the state-of-the-arts by a large margin.
• Score: 24.23826907954389
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural implicit surface learning has shown significant progress in multi-view 3D reconstruction, where an object is represented by multilayer perceptrons that provide continuous implicit surface representation and view-dependent radiance. However, current methods often fail to accurately reconstruct reflective surfaces, leading to severe ambiguity. To overcome this issue, we propose Ref-NeuS, which aims to reduce ambiguity by attenuating the effect of reflective surfaces. Specifically, we utilize an anomaly detector to estimate an explicit reflection score with the guidance of multi-view context to localize reflective surfaces. Afterward, we design a reflection-aware photometric loss that adaptively reduces ambiguity by modeling rendered color as a Gaussian distribution, with the reflection score representing the variance. We show that together with a reflection direction-dependent radiance, our model achieves high-quality surface reconstruction on reflective surfaces and outperforms the state-of-the-arts by a large margin. Besides, our model is also comparable on general surfaces.

Related papers

• NeRSP: Neural 3D Reconstruction for Reflective Objects with Sparse Polarized Images [62.752710734332894]
  NeRSP is a Neural 3D reconstruction technique for Reflective surfaces with Sparse Polarized images. We derive photometric and geometric cues from the polarimetric image formation model and multiview azimuth consistency. We achieve the state-of-the-art surface reconstruction results with only 6 views as input.
  arXiv  Detail & Related papers  (2024-06-11T09:53:18Z)
• Monocular Identity-Conditioned Facial Reflectance Reconstruction [71.90507628715388]
  Existing methods rely on a large amount of light-stage captured data to learn facial reflectance models. We learn the reflectance prior in image space rather than UV space and present a framework named ID2Reflectance. Our framework can directly estimate the reflectance maps of a single image while using limited reflectance data for training.
  arXiv  Detail & Related papers  (2024-03-30T09:43:40Z)
• UniSDF: Unifying Neural Representations for High-Fidelity 3D Reconstruction of Complex Scenes with Reflections [92.38975002642455]
  We propose UniSDF, a general purpose 3D reconstruction method that can reconstruct large complex scenes with reflections. Our method is able to robustly reconstruct complex large-scale scenes with fine details and reflective surfaces.
  arXiv  Detail & Related papers  (2023-12-20T18:59:42Z)
• Diffusion Reflectance Map: Single-Image Stochastic Inverse Rendering of Illumination and Reflectance [19.20790327389337]
  Reflectance bounds the frequency spectrum of illumination in the object appearance. We introduce the first inverse rendering method, which recovers the attenuated frequency spectrum of an illumination jointly with the reflectance of an object of known geometry.
  arXiv  Detail & Related papers  (2023-12-07T18:50:00Z)
• TraM-NeRF: Tracing Mirror and Near-Perfect Specular Reflections through Neural Radiance Fields [3.061835990893184]
  Implicit representations like Neural Radiance Fields (NeRF) showed impressive results for rendering of complex scenes with fine details. We present a novel reflection tracing method tailored for the involved volume rendering within NeRF. We derive efficient strategies for importance sampling and the transmittance computation along rays from only few samples.
  arXiv  Detail & Related papers  (2023-10-16T17:59:56Z)
• Predicting Surface Reflectance Properties of Outdoor Scenes Under Unknown Natural Illumination [6.767885381740952]
  This paper proposes a complete framework to predict surface reflectance properties of outdoor scenes under unknown natural illumination. We recast the problem into its two constituent components involving the BRDF incoming light and outgoing view directions. We present experiments that show that rendering with the predicted reflectance properties results in a visually similar appearance to using textures.
  arXiv  Detail & Related papers  (2021-05-14T13:31:47Z)
• Location-aware Single Image Reflection Removal [54.93808224890273]
  This paper proposes a novel location-aware deep learning-based single image reflection removal method. We use a reflection confidence map as the cues for the network to learn how to encode the reflection information adaptively. The integration of location information into the network significantly improves the quality of reflection removal results.
  arXiv  Detail & Related papers  (2020-12-13T19:34:35Z)
• Two-Stage Single Image Reflection Removal with Reflection-Aware Guidance [78.34235841168031]
  We present a novel two-stage network with reflection-aware guidance (RAGNet) for single image reflection removal (SIRR) RAG can be used (i) to mitigate the effect of reflection from the observation, and (ii) to generate mask in partial convolution for mitigating the effect of deviating from linear combination hypothesis. Experiments on five commonly used datasets demonstrate the quantitative and qualitative superiority of our RAGNet in comparison to the state-of-the-art SIRR methods.
  arXiv  Detail & Related papers  (2020-12-02T03:14:57Z)
• Monocular Reconstruction of Neural Face Reflectance Fields [0.0]
  The reflectance field of a face describes the reflectance properties responsible for complex lighting effects. Most existing methods for estimating the face reflectance from a monocular image assume faces to be diffuse with very few approaches adding a specular component. We present a new neural representation for face reflectance where we can estimate all components of the reflectance responsible for the final appearance from a single monocular image.
  arXiv  Detail & Related papers  (2020-08-24T08:19:05Z)
• Polarized Reflection Removal with Perfect Alignment in the Wild [66.48211204364142]
  We present a novel formulation to removing reflection from polarized images in the wild. We first identify the misalignment issues of existing reflection removal datasets. We build a new dataset with more than 100 types of glass in which obtained transmission images are perfectly aligned with input mixed images.
  arXiv  Detail & Related papers  (2020-03-28T13:29:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.