NeIF: Representing General Reflectance as Neural Intrinsics Fields for Uncalibrated Photometric Stereo

• URL: http://arxiv.org/abs/2208.08897v2
• Date: Fri, 19 Aug 2022 09:45:43 GMT
• Title: NeIF: Representing General Reflectance as Neural Intrinsics Fields for Uncalibrated Photometric Stereo
• Authors: Zongrui Li, Qian Zheng, Feishi Wang, Boxin Shi, Gang Pan, Xudong Jiang
• Abstract summary: Uncalibrated photometric stereo (UPS) is challenging due to the inherent ambiguity brought by unknown light. This paper establishes an implicit relation between light clues and light estimation and solves UPS in an unsupervised manner.
• Score: 70.71400320657035
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Uncalibrated photometric stereo (UPS) is challenging due to the inherent ambiguity brought by unknown light. Existing solutions alleviate the ambiguity by either explicitly associating reflectance to light conditions or resolving light conditions in a supervised manner. This paper establishes an implicit relation between light clues and light estimation and solves UPS in an unsupervised manner. The key idea is to represent the reflectance as four neural intrinsics fields, i.e., position, light, specular, and shadow, based on which the neural light field is implicitly associated with light clues of specular reflectance and cast shadow. The unsupervised, joint optimization of neural intrinsics fields can be free from training data bias as well as accumulating error, and fully exploits all observed pixel values for UPS. Our method achieves a superior performance advantage over state-of-the-art UPS methods on public and self-collected datasets, under regular and challenging setups. The code will be released soon.

Related papers

• Generalizable Non-Line-of-Sight Imaging with Learnable Physical Priors [52.195637608631955]
  Non-line-of-sight (NLOS) imaging has attracted increasing attention due to its potential applications. Existing NLOS reconstruction approaches are constrained by the reliance on empirical physical priors. We introduce a novel learning-based solution, comprising two key designs: Learnable Path Compensation (LPC) and Adaptive Phasor Field (APF)
  arXiv  Detail & Related papers  (2024-09-21T04:39:45Z)
• Spin-UP: Spin Light for Natural Light Uncalibrated Photometric Stereo [68.90962272268793]
  Natural Light Uncalibrated Photometric Stereo (NaUPS) relieves the strict environment and light assumptions in classical Uncalibrated Photometric Stereo (UPS) methods. Existing works impose strong assumptions on the environment lights and objects' material, restricting the effectiveness in more general scenarios. We propose Spin Light Uncalibrated Photometric Stereo (Spin-UP), an unsupervised method to tackle NaUPS in various environment lights and objects.
  arXiv  Detail & Related papers  (2024-04-02T03:29:23Z)
• Aleth-NeRF: Illumination Adaptive NeRF with Concealing Field Assumption [65.96818069005145]
  We introduce the concept of a "Concealing Field," which assigns transmittance values to the surrounding air to account for illumination effects. In dark scenarios, we assume that object emissions maintain a standard lighting level but are attenuated as they traverse the air during the rendering process. We present a comprehensive multi-view dataset captured under challenging illumination conditions for evaluation.
  arXiv  Detail & Related papers  (2023-12-14T16:24:09Z)
• WildLight: In-the-wild Inverse Rendering with a Flashlight [77.31815397135381]
  We propose a practical photometric solution for in-the-wild inverse rendering under unknown ambient lighting. Our system recovers scene geometry and reflectance using only multi-view images captured by a smartphone. We demonstrate by extensive experiments that our method is easy to implement, casual to set up, and consistently outperforms existing in-the-wild inverse rendering techniques.
  arXiv  Detail & Related papers  (2023-03-24T17:59:56Z)
• Self-calibrating Photometric Stereo by Neural Inverse Rendering [88.67603644930466]
  This paper tackles the task of uncalibrated photometric stereo for 3D object reconstruction. We propose a new method that jointly optimize object shape, light directions, and light intensities. Our method demonstrates state-of-the-art accuracy in light estimation and shape recovery on real-world datasets.
  arXiv  Detail & Related papers  (2022-07-16T02:46:15Z)
• Unsupervised Low-Light Image Enhancement via Histogram Equalization Prior [40.61944814314655]
  We propose an unsupervised low-light image enhancement method based on an effective prior histogram termed equalization prior (HEP) We introduce a Noise Disentanglement Module (NDM) to disentangle the noise and content in the reflectance maps with the reliable aid of unpaired clean images. Our method performs favorably against the state-of-the-art unsupervised low-light enhancement algorithms and even matches the state-of-the-art supervised algorithms.
  arXiv  Detail & Related papers  (2021-12-03T07:51:08Z)
• A New Dimension in Testimony: Relighting Video with Reflectance Field Exemplars [1.069384486725302]
  We present a learning-based method for estimating 4D reflectance field of a person given video footage illuminated under a flat-lit environment of the same subject. We estimate the lighting environment of the input video footage and use the subject's reflectance field to create synthetic images of the subject illuminated by the input lighting environment. We evaluate our method on the video footage of the real Holocaust survivors and show that our method outperforms the state-of-the-art methods in both realism and speed.
  arXiv  Detail & Related papers  (2021-04-06T20:29:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.