A Simple Approach to Differentiable Rendering of SDFs

• URL: http://arxiv.org/abs/2405.08733v2
• Date: Fri, 7 Jun 2024 08:31:20 GMT
• Title: A Simple Approach to Differentiable Rendering of SDFs
• Authors: Zichen Wang, Xi Deng, Ziyi Zhang, Wenzel Jakob, Steve Marschner,
• Abstract summary: We present a simple algorithm for differentiable rendering of surfaces represented by Signed Distance Fields (SDF) Our method expands the lower-dimensional boundary integral into a thin band that is easy to sample when the underlying surface is represented by an SDF.
• Score: 21.97043707520229
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a simple algorithm for differentiable rendering of surfaces represented by Signed Distance Fields (SDF), which makes it easy to integrate rendering into gradient-based optimization pipelines. To tackle visibility-related derivatives that make rendering non-differentiable, existing physically based differentiable rendering methods often rely on elaborate guiding data structures or reparameterization with a global impact on variance. In this article, we investigate an alternative that embraces nonzero bias in exchange for low variance and architectural simplicity. Our method expands the lower-dimensional boundary integral into a thin band that is easy to sample when the underlying surface is represented by an SDF. We demonstrate the performance and robustness of our formulation in end-to-end inverse rendering tasks, where it obtains results that are competitive with or superior to existing work.

Related papers

• Efficient Fairness-Performance Pareto Front Computation [51.558848491038916]
  We show that optimal fair representations possess several useful structural properties. We then show that these approxing problems can be solved efficiently via concave programming methods.
  arXiv  Detail & Related papers  (2024-09-26T08:46:48Z)
• Probabilistic Directed Distance Fields for Ray-Based Shape Representations [8.134429779950658]
  Directed Distance Fields (DDFs) are a novel neural shape representation that builds upon classical distance fields. We show how to model inherent discontinuities in the underlying field. We then apply DDFs to several applications, including single-shape fitting, generative modelling, and single-image 3D reconstruction.
  arXiv  Detail & Related papers  (2024-04-13T21:02:49Z)
• NeuS-PIR: Learning Relightable Neural Surface using Pre-Integrated Rendering [23.482941494283978]
  This paper presents a method, namely NeuS-PIR, for recovering relightable neural surfaces from multi-view images or video. Unlike methods based on NeRF and discrete meshes, our method utilizes implicit neural surface representation to reconstruct high-quality geometry. Our method enables advanced applications such as relighting, which can be seamlessly integrated with modern graphics engines.
  arXiv  Detail & Related papers  (2023-06-13T09:02:57Z)
• Reflected Diffusion Models [93.26107023470979]
  We present Reflected Diffusion Models, which reverse a reflected differential equation evolving on the support of the data. Our approach learns the score function through a generalized score matching loss and extends key components of standard diffusion models.
  arXiv  Detail & Related papers  (2023-04-10T17:54:38Z)
• Editing Out-of-domain GAN Inversion via Differential Activations [56.62964029959131]
  We propose a novel GAN prior based editing framework to tackle the out-of-domain inversion problem with a composition-decomposition paradigm. With the aid of the generated Diff-CAM mask, a coarse reconstruction can intuitively be composited by the paired original and edited images. In the decomposition phase, we further present a GAN prior based deghosting network for separating the final fine edited image from the coarse reconstruction.
  arXiv  Detail & Related papers  (2022-07-17T10:34:58Z)
• Differentiable Rendering of Neural SDFs through Reparameterization [32.47993049026182]
  We present a method to automatically compute correct gradients with respect to geometric scene parameters in neural SDFs. Our approach builds on area-sampling techniques and develops a continuous warping function for SDFs to account for discontinuities. Our differentiable can be used to optimize neural shapes from multi-view images and produces comparable 3D reconstructions.
  arXiv  Detail & Related papers  (2022-06-10T20:30:26Z)
• RISP: Rendering-Invariant State Predictor with Differentiable Simulation and Rendering for Cross-Domain Parameter Estimation [110.4255414234771]
  Existing solutions require massive training data or lack generalizability to unknown rendering configurations. We propose a novel approach that marries domain randomization and differentiable rendering gradients to address this problem. Our approach achieves significantly lower reconstruction errors and has better generalizability among unknown rendering configurations.
  arXiv  Detail & Related papers  (2022-05-11T17:59:51Z)
• Representing 3D Shapes with Probabilistic Directed Distance Fields [7.528141488548544]
  We develop a novel shape representation that allows fast differentiable rendering within an implicit architecture. We show how to model inherent discontinuities in the underlying field. We also apply our method to fitting single shapes, unpaired 3D-aware generative image modelling, and single-image 3D reconstruction tasks.
  arXiv  Detail & Related papers  (2021-12-10T02:15:47Z)
• Gradient-SDF: A Semi-Implicit Surface Representation for 3D Reconstruction [53.315347543761426]
  Gradient-SDF is a novel representation for 3D geometry that combines the advantages of implict and explicit representations. By storing at every voxel both the signed distance field as well as its gradient vector field, we enhance the capability of implicit representations. We show that (1) the Gradient-SDF allows us to perform direct SDF tracking from depth images, using efficient storage schemes like hash maps, and that (2) the Gradient-SDF representation enables us to perform photometric bundle adjustment directly in a voxel representation.
  arXiv  Detail & Related papers  (2021-11-26T18:33:14Z)
• Differentiable Surface Rendering via Non-Differentiable Sampling [19.606523934811577]
  We present a method for differentiable rendering of 3D surfaces that supports both explicit and implicit representations. We show for the first time efficient, differentiable rendering of an iso extracted from a neural radiance field (NeRF), and demonstrate surface-based, rather than volume-based, rendering of a NeRF.
  arXiv  Detail & Related papers  (2021-08-10T19:25:06Z)
• Invariant Integration in Deep Convolutional Feature Space [77.99182201815763]
  We show how to incorporate prior knowledge to a deep neural network architecture in a principled manner. We report state-of-the-art performance on the Rotated-MNIST dataset.
  arXiv  Detail & Related papers  (2020-04-20T09:45:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.