Plateau-reduced Differentiable Path Tracing
- URL: http://arxiv.org/abs/2211.17263v2
- Date: Tue, 28 Mar 2023 14:41:01 GMT
- Title: Plateau-reduced Differentiable Path Tracing
- Authors: Michael Fischer, Tobias Ritschel
- Abstract summary: We show that inverse rendering might not converge due to inherent plateaus, i.e. regions of zero gradient, in the objective function.
We propose to alleviate this by convolving the high-dimensional rendering function that maps parameters to images with an additional kernel that blurs the parameter space.
- Score: 18.174063717952187
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Current differentiable renderers provide light transport gradients with
respect to arbitrary scene parameters. However, the mere existence of these
gradients does not guarantee useful update steps in an optimization. Instead,
inverse rendering might not converge due to inherent plateaus, i.e., regions of
zero gradient, in the objective function. We propose to alleviate this by
convolving the high-dimensional rendering function that maps scene parameters
to images with an additional kernel that blurs the parameter space. We describe
two Monte Carlo estimators to compute plateau-free gradients efficiently, i.e.,
with low variance, and show that these translate into net-gains in optimization
error and runtime performance. Our approach is a straightforward extension to
both black-box and differentiable renderers and enables optimization of
problems with intricate light transport, such as caustics or global
illumination, that existing differentiable renderers do not converge on.
Related papers
- Locally Orderless Images for Optimization in Differentiable Rendering [80.09571356394574]
We introduce a method that uses locally orderless images, where each pixel maps to a histogram of intensities that preserves local variations in appearance.
We validate our method on various inverse problems using both synthetic and real data.
arXiv Detail & Related papers (2025-03-27T19:17:58Z) - Radiant Foam: Real-Time Differentiable Ray Tracing [32.66224278608794]
We propose a novel scene representation which avoids approximations, but keeps the efficiency and reconstruction quality of splatting.
We leverage a decades-old efficient mesh ray tracing algorithm which has been largely overlooked in recent computer vision research.
arXiv Detail & Related papers (2025-02-03T08:49:57Z) - Flash Cache: Reducing Bias in Radiance Cache Based Inverse Rendering [62.92985004295714]
We present a method that avoids approximations that introduce bias into the renderings and, more importantly, the gradients used for optimization.
We show that by removing these biases our approach improves the generality of radiance cache based inverse rendering, as well as increasing quality in the presence of challenging light transport effects such as specular reflections.
arXiv Detail & Related papers (2024-09-09T17:59:57Z) - Zero Grads: Learning Local Surrogate Losses for Non-Differentiable Graphics [13.558059987555234]
We show how to automatically optimize graphics for undefined or zero gradients.
The fitting is performed online, alongside the parameter optimization, and self-supervised, without pre-computed data or pre-trained models.
In contrast to other algorithms, our approach scales well to higher dimensions, which we demonstrate on problems with up to 35k variables.
arXiv Detail & Related papers (2023-08-10T17:57:22Z) - Optimization using Parallel Gradient Evaluations on Multiple Parameters [51.64614793990665]
We propose a first-order method for convex optimization, where gradients from multiple parameters can be used during each step of gradient descent.
Our method uses gradients from multiple parameters in synergy to update these parameters together towards the optima.
arXiv Detail & Related papers (2023-02-06T23:39:13Z) - RISP: Rendering-Invariant State Predictor with Differentiable Simulation
and Rendering for Cross-Domain Parameter Estimation [110.4255414234771]
Existing solutions require massive training data or lack generalizability to unknown rendering configurations.
We propose a novel approach that marries domain randomization and differentiable rendering gradients to address this problem.
Our approach achieves significantly lower reconstruction errors and has better generalizability among unknown rendering configurations.
arXiv Detail & Related papers (2022-05-11T17:59:51Z) - On Training Implicit Models [75.20173180996501]
We propose a novel gradient estimate for implicit models, named phantom gradient, that forgoes the costly computation of the exact gradient.
Experiments on large-scale tasks demonstrate that these lightweight phantom gradients significantly accelerate the backward passes in training implicit models by roughly 1.7 times.
arXiv Detail & Related papers (2021-11-09T14:40:24Z) - Differentiable Rendering with Perturbed Optimizers [85.66675707599782]
Reasoning about 3D scenes from their 2D image projections is one of the core problems in computer vision.
Our work highlights the link between some well-known differentiable formulations and randomly smoothed renderings.
We apply our method to 3D scene reconstruction and demonstrate its advantages on the tasks of 6D pose estimation and 3D mesh reconstruction.
arXiv Detail & Related papers (2021-10-18T08:56:23Z) - Efficient and Differentiable Shadow Computation for Inverse Problems [64.70468076488419]
Differentiable geometric computation has received increasing interest for image-based inverse problems.
We propose an efficient yet efficient approach for differentiable visibility and soft shadow computation.
As our formulation is differentiable, it can be used to solve inverse problems such as texture, illumination, rigid pose, and deformation recovery from images.
arXiv Detail & Related papers (2021-04-01T09:29:05Z) - Effect of barren plateaus on gradient-free optimization [0.755972004983746]
Barren plateau landscapes correspond to that vanish exponentially in the number of qubits.
Such landscapes have been demonstrated for variational quantum algorithms and quantum neural networks with either deep circuits or global cost functions.
arXiv Detail & Related papers (2020-11-24T17:41:13Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.