Neural Reflectance for Shape Recovery with Shadow Handling

• URL: http://arxiv.org/abs/2203.12909v1
• Date: Thu, 24 Mar 2022 07:57:20 GMT
• Title: Neural Reflectance for Shape Recovery with Shadow Handling
• Authors: Junxuan Li and Hongdong Li
• Abstract summary: This paper aims at recovering the shape of a scene with unknown, non-Lambertian, and possibly spatially-varying surface materials. We propose a coordinate-based deep reflectance (multilayer perceptron) to parameterize both the unknown 3D shape and the unknown at every surface point. This network is able to leverage the observed photometric variance and shadows on the surface, and recover both surface shape and general non-Lambertian reflectance.
• Score: 88.67603644930466
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper aims at recovering the shape of a scene with unknown, non-Lambertian, and possibly spatially-varying surface materials. When the shape of the object is highly complex and that shadows cast on the surface, the task becomes very challenging. To overcome these challenges, we propose a coordinate-based deep MLP (multilayer perceptron) to parameterize both the unknown 3D shape and the unknown reflectance at every surface point. This network is able to leverage the observed photometric variance and shadows on the surface, and recover both surface shape and general non-Lambertian reflectance. We explicitly predict cast shadows, mitigating possible artifacts on these shadowing regions, leading to higher estimation accuracy. Our framework is entirely self-supervised, in the sense that it requires neither ground truth shape nor BRDF. Tests on real-world images demonstrate that our method outperform existing methods by a significant margin. Thanks to the small size of the MLP-net, our method is an order of magnitude faster than previous CNN-based methods.

Related papers

• DeepShaRM: Multi-View Shape and Reflectance Map Recovery Under Unknown Lighting [35.18426818323455]
  We derive a novel multi-view method, DeepShaRM, that achieves state-of-the-art accuracy on this challenging task. We introduce a novel deep reflectance map estimation network that recovers the camera-view reflectance maps. A deep shape-from-shading network then updates the geometry estimate expressed with a signed distance function.
  arXiv  Detail & Related papers  (2023-10-26T17:50:10Z)
• NeTO:Neural Reconstruction of Transparent Objects with Self-Occlusion Aware Refraction-Tracing [44.22576861939435]
  We present a novel method, called NeTO, for capturing 3D geometry of solid transparent objects from 2D images via volume rendering. Our method achieves faithful reconstruction results and outperforms prior works by a large margin.
  arXiv  Detail & Related papers  (2023-03-20T15:50:00Z)
• Self-calibrating Photometric Stereo by Neural Inverse Rendering [88.67603644930466]
  This paper tackles the task of uncalibrated photometric stereo for 3D object reconstruction. We propose a new method that jointly optimize object shape, light directions, and light intensities. Our method demonstrates state-of-the-art accuracy in light estimation and shape recovery on real-world datasets.
  arXiv  Detail & Related papers  (2022-07-16T02:46:15Z)
• SNeS: Learning Probably Symmetric Neural Surfaces from Incomplete Data [77.53134858717728]
  We build on the strengths of recent advances in neural reconstruction and rendering such as Neural Radiance Fields (NeRF) We apply a soft symmetry constraint to the 3D geometry and material properties, having factored appearance into lighting, albedo colour and reflectivity. We show that it can reconstruct unobserved regions with high fidelity and render high-quality novel view images.
  arXiv  Detail & Related papers  (2022-06-13T17:37:50Z)
• DeepShadow: Neural Shape from Shadow [12.283891012446647]
  DeepShadow is a one-shot method for recovering the depth map and surface normals from photometric stereo shadow maps. We show that the self and cast shadows not only do not disturb 3D reconstruction, but can be used alone, as a strong learning signal. Our method is the first to reconstruct 3D shape-from-shadows using neural networks.
  arXiv  Detail & Related papers  (2022-03-28T20:11:15Z)
• AvatarMe++: Facial Shape and BRDF Inference with Photorealistic Rendering-Aware GANs [119.23922747230193]
  We introduce the first method that is able to reconstruct render-ready 3D facial geometry and BRDF from a single "in-the-wild" image. Our method outperforms the existing arts by a significant margin and reconstructs high-resolution 3D faces from a single low-resolution image.
  arXiv  Detail & Related papers  (2021-12-11T11:36:30Z)
• NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination [60.89737319987051]
  We address the problem of recovering shape and spatially-varying reflectance of an object from posed multi-view images of the object illuminated by one unknown lighting condition. This enables the rendering of novel views of the object under arbitrary environment lighting and editing of the object's material properties.
  arXiv  Detail & Related papers  (2021-06-03T16:18:01Z)
• Multi-view 3D Reconstruction of a Texture-less Smooth Surface of Unknown Generic Reflectance [86.05191217004415]
  Multi-view reconstruction of texture-less objects with unknown surface reflectance is a challenging task. This paper proposes a simple and robust solution to this problem based on a co-light scanner.
  arXiv  Detail & Related papers  (2021-05-25T01:28:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.