Invertible Neural BRDF for Object Inverse Rendering

• URL: http://arxiv.org/abs/2008.04030v2
• Date: Tue, 11 Aug 2020 04:15:00 GMT
• Title: Invertible Neural BRDF for Object Inverse Rendering
• Authors: Zhe Chen, Shohei Nobuhara, and Ko Nishino
• Abstract summary: We introduce a novel neural network-based BRDF model and a Bayesian framework for object inverse rendering. We experimentally validate the accuracy of the invertible neural BRDF model on a large number of measured data. Results show new ways in which deep neural networks can help solve challenging inverse problems.
• Score: 27.86441556552318
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce a novel neural network-based BRDF model and a Bayesian framework for object inverse rendering, i.e., joint estimation of reflectance and natural illumination from a single image of an object of known geometry. The BRDF is expressed with an invertible neural network, namely, normalizing flow, which provides the expressive power of a high-dimensional representation, computational simplicity of a compact analytical model, and physical plausibility of a real-world BRDF. We extract the latent space of real-world reflectance by conditioning this model, which directly results in a strong reflectance prior. We refer to this model as the invertible neural BRDF model (iBRDF). We also devise a deep illumination prior by leveraging the structural bias of deep neural networks. By integrating this novel BRDF model and reflectance and illumination priors in a MAP estimation formulation, we show that this joint estimation can be computed efficiently with stochastic gradient descent. We experimentally validate the accuracy of the invertible neural BRDF model on a large number of measured data and demonstrate its use in object inverse rendering on a number of synthetic and real images. The results show new ways in which deep neural networks can help solve challenging radiometric inverse problems.

Related papers

• Physically Based Neural Bidirectional Reflectance Distribution Function [40.726751711127946]
  We introduce the physically based neural bidirectional reflectance distribution function (PBNBRDF) Our model accurately reconstructs real-world materials while uniquely enforcing physical properties for realistic BRDFs.
  arXiv  Detail & Related papers  (2024-11-04T18:17:44Z)
• NeRF-Casting: Improved View-Dependent Appearance with Consistent Reflections [57.63028964831785]
  Recent works have improved NeRF's ability to render detailed specular appearance of distant environment illumination, but are unable to synthesize consistent reflections of closer content. We address these issues with an approach based on ray tracing. Instead of querying an expensive neural network for the outgoing view-dependent radiance at points along each camera ray, our model casts rays from these points and traces them through the NeRF representation to render feature vectors.
  arXiv  Detail & Related papers  (2024-05-23T17:59:57Z)
• NePF: Neural Photon Field for Single-Stage Inverse Rendering [6.977356702921476]
  We present a novel single-stage framework, Neural Photon Field (NePF), to address the ill-posed inverse rendering from multi-view images. NePF achieves this unification by fully utilizing the physical implication behind the weight function of neural implicit surfaces. We evaluate our method on both real and synthetic datasets.
  arXiv  Detail & Related papers  (2023-11-20T06:15:46Z)
• TensoIR: Tensorial Inverse Rendering [51.57268311847087]
  TensoIR is a novel inverse rendering approach based on tensor factorization and neural fields. TensoRF is a state-of-the-art approach for radiance field modeling.
  arXiv  Detail & Related papers  (2023-04-24T21:39:13Z)
• Physics-based Indirect Illumination for Inverse Rendering [70.27534648770057]
  We present a physics-based inverse rendering method that learns the illumination, geometry, and materials of a scene from posed multi-view RGB images. As a side product, our physics-based inverse rendering model also facilitates flexible and realistic material editing as well as relighting.
  arXiv  Detail & Related papers  (2022-12-09T07:33:49Z)
• IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis [90.03590032170169]
  We present intrinsic neural radiance fields, dubbed IntrinsicNeRF, which introduce intrinsic decomposition into the NeRF-based neural rendering method. Our experiments and editing samples on both object-specific/room-scale scenes and synthetic/real-word data demonstrate that we can obtain consistent intrinsic decomposition results.
  arXiv  Detail & Related papers  (2022-10-02T22:45:11Z)
• Neural apparent BRDF fields for multiview photometric stereo [22.11062920603769]
  We propose to tackle the multiview photometric stereo problem using an extension of Neural Radiance Fields (NeRFs) The geometric part of our neural representation predicts surface normal direction, allowing us to reason about local surface reflectance. We demonstrate our approach on a multiview photometric stereo benchmark and show that competitive performance can be obtained with the neural density representation of a NeRF.
  arXiv  Detail & Related papers  (2022-07-14T10:16:25Z)
• Robust lEarned Shrinkage-Thresholding (REST): Robust unrolling for sparse recover [87.28082715343896]
  We consider deep neural networks for solving inverse problems that are robust to forward model mis-specifications. We design a new robust deep neural network architecture by applying algorithm unfolding techniques to a robust version of the underlying recovery problem. The proposed REST network is shown to outperform state-of-the-art model-based and data-driven algorithms in both compressive sensing and radar imaging problems.
  arXiv  Detail & Related papers  (2021-10-20T06:15:45Z)
• Modeling the Nonsmoothness of Modern Neural Networks [35.93486244163653]
  We quantify the nonsmoothness using a feature named the sum of the magnitude of peaks (SMP) We envision that the nonsmoothness feature can potentially be used as a forensic tool for regression-based applications of neural networks.
  arXiv  Detail & Related papers  (2021-03-26T20:55:19Z)
• Neural BRDF Representation and Importance Sampling [79.84316447473873]
  We present a compact neural network-based representation of reflectance BRDF data. We encode BRDFs as lightweight networks, and propose a training scheme with adaptive angular sampling. We evaluate encoding results on isotropic and anisotropic BRDFs from multiple real-world datasets.
  arXiv  Detail & Related papers  (2021-02-11T12:00:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.