Flash Cache: Reducing Bias in Radiance Cache Based Inverse Rendering

• URL: http://arxiv.org/abs/2409.05867v1
• Date: Mon, 9 Sep 2024 17:59:57 GMT
• Title: Flash Cache: Reducing Bias in Radiance Cache Based Inverse Rendering
• Authors: Benjamin Attal, Dor Verbin, Ben Mildenhall, Peter Hedman, Jonathan T. Barron, Matthew O'Toole, Pratul P. Srinivasan,
• Abstract summary: We present a method that avoids approximations that introduce bias into the renderings and, more importantly, the gradients used for optimization. We show that by removing these biases our approach improves the generality of radiance cache based inverse rendering, as well as increasing quality in the presence of challenging light transport effects such as specular reflections.
• Score: 62.92985004295714
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: State-of-the-art techniques for 3D reconstruction are largely based on volumetric scene representations, which require sampling multiple points to compute the color arriving along a ray. Using these representations for more general inverse rendering -- reconstructing geometry, materials, and lighting from observed images -- is challenging because recursively path-tracing such volumetric representations is expensive. Recent works alleviate this issue through the use of radiance caches: data structures that store the steady-state, infinite-bounce radiance arriving at any point from any direction. However, these solutions rely on approximations that introduce bias into the renderings and, more importantly, into the gradients used for optimization. We present a method that avoids these approximations while remaining computationally efficient. In particular, we leverage two techniques to reduce variance for unbiased estimators of the rendering equation: (1) an occlusion-aware importance sampler for incoming illumination and (2) a fast cache architecture that can be used as a control variate for the radiance from a high-quality, but more expensive, volumetric cache. We show that by removing these biases our approach improves the generality of radiance cache based inverse rendering, as well as increasing quality in the presence of challenging light transport effects such as specular reflections.

Related papers

• CVT-xRF: Contrastive In-Voxel Transformer for 3D Consistent Radiance Fields from Sparse Inputs [65.80187860906115]
  We propose a novel approach to improve NeRF's performance with sparse inputs. We first adopt a voxel-based ray sampling strategy to ensure that the sampled rays intersect with a certain voxel in 3D space. We then randomly sample additional points within the voxel and apply a Transformer to infer the properties of other points on each ray, which are then incorporated into the volume rendering.
  arXiv  Detail & Related papers  (2024-03-25T15:56:17Z)
• GIR: 3D Gaussian Inverse Rendering for Relightable Scene Factorization [62.13932669494098]
  This paper presents a 3D Gaussian Inverse Rendering (GIR) method, employing 3D Gaussian representations to factorize the scene into material properties, light, and geometry. We compute the normal of each 3D Gaussian using the shortest eigenvector, with a directional masking scheme forcing accurate normal estimation without external supervision. We adopt an efficient voxel-based indirect illumination tracing scheme that stores direction-aware outgoing radiance in each 3D Gaussian to disentangle secondary illumination for approximating multi-bounce light transport.
  arXiv  Detail & Related papers  (2023-12-08T16:05:15Z)
• Inverse Global Illumination using a Neural Radiometric Prior [26.29610954064107]
  Inverse rendering methods that account for global illumination are becoming more popular. This paper proposes a radiometric prior as a simple alternative to building complete path integrals in a traditional differentiable path tracer.
  arXiv  Detail & Related papers  (2023-05-03T15:36:39Z)
• Factorized Inverse Path Tracing for Efficient and Accurate Material-Lighting Estimation [97.0195314255101]
  Inverse path tracing is expensive to compute, and ambiguities exist between reflection and emission. Our Factorized Inverse Path Tracing (FIPT) addresses these challenges by using a factored light transport formulation. Our algorithm enables accurate material and lighting optimization faster than previous work, and is more effective at resolving ambiguities.
  arXiv  Detail & Related papers  (2023-04-12T07:46:05Z)
• Differentiable Rendering with Reparameterized Volume Sampling [2.717399369766309]
  In view synthesis, a neural radiance field approximates underlying density and radiance fields based on a sparse set of scene pictures. This rendering algorithm is fully differentiable and facilitates gradient-based optimization of the fields. We propose a simple end-to-end differentiable sampling algorithm based on inverse transform sampling.
  arXiv  Detail & Related papers  (2023-02-21T19:56:50Z)
• Differentiable Rendering with Perturbed Optimizers [85.66675707599782]
  Reasoning about 3D scenes from their 2D image projections is one of the core problems in computer vision. Our work highlights the link between some well-known differentiable formulations and randomly smoothed renderings. We apply our method to 3D scene reconstruction and demonstrate its advantages on the tasks of 6D pose estimation and 3D mesh reconstruction.
  arXiv  Detail & Related papers  (2021-10-18T08:56:23Z)
• Efficient and Differentiable Shadow Computation for Inverse Problems [64.70468076488419]
  Differentiable geometric computation has received increasing interest for image-based inverse problems. We propose an efficient yet efficient approach for differentiable visibility and soft shadow computation. As our formulation is differentiable, it can be used to solve inverse problems such as texture, illumination, rigid pose, and deformation recovery from images.
  arXiv  Detail & Related papers  (2021-04-01T09:29:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.