A Theory of Topological Derivatives for Inverse Rendering of Geometry

• URL: http://arxiv.org/abs/2308.09865v1
• Date: Sat, 19 Aug 2023 00:55:55 GMT
• Title: A Theory of Topological Derivatives for Inverse Rendering of Geometry
• Authors: Ishit Mehta, Manmohan Chandraker, Ravi Ramamoorthi
• Abstract summary: We introduce a theoretical framework for differentiable surface evolution that allows discrete topology changes through the use of topological derivatives. We validate the proposed theory with optimization of closed curves in 2D and surfaces in 3D to lend insights into limitations of current methods.
• Score: 87.49881303178061
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce a theoretical framework for differentiable surface evolution that allows discrete topology changes through the use of topological derivatives for variational optimization of image functionals. While prior methods for inverse rendering of geometry rely on silhouette gradients for topology changes, such signals are sparse. In contrast, our theory derives topological derivatives that relate the introduction of vanishing holes and phases to changes in image intensity. As a result, we enable differentiable shape perturbations in the form of hole or phase nucleation. We validate the proposed theory with optimization of closed curves in 2D and surfaces in 3D to lend insights into limitations of current methods and enable improved applications such as image vectorization, vector-graphics generation from text prompts, single-image reconstruction of shape ambigrams and multi-view 3D reconstruction.

Related papers

• GeoGS3D: Single-view 3D Reconstruction via Geometric-aware Diffusion Model and Gaussian Splatting [81.03553265684184]
  We introduce GeoGS3D, a framework for reconstructing detailed 3D objects from single-view images. We propose a novel metric, Gaussian Divergence Significance (GDS), to prune unnecessary operations during optimization. Experiments demonstrate that GeoGS3D generates images with high consistency across views and reconstructs high-quality 3D objects.
  arXiv  Detail & Related papers  (2024-03-15T12:24:36Z)
• Physics-informed neural networks for transformed geometries and manifolds [0.0]
  We propose a novel method for integrating geometric transformations within PINNs to robustly accommodate geometric variations. We demonstrate the enhanced flexibility over traditional PINNs, especially under geometric variations. The proposed framework presents an outlook for training deep neural operators over parametrized geometries.
  arXiv  Detail & Related papers  (2023-11-27T15:47:33Z)
• Deep neural networks on diffeomorphism groups for optimal shape reparameterization [44.99833362998488]
  We propose an algorithm for constructing approximations of orientation-preserving diffeomorphisms by composition of elementary diffeomorphisms. The algorithm is implemented using PyTorch, and is applicable for both unparametrized curves and surfaces.
  arXiv  Detail & Related papers  (2022-07-22T15:25:59Z)
• A Level Set Theory for Neural Implicit Evolution under Explicit Flows [102.18622466770114]
  Coordinate-based neural networks parameterizing implicit surfaces have emerged as efficient representations of geometry. We present a framework that allows applying deformation operations defined for triangle meshes onto such implicit surfaces. We show that our approach exhibits improvements for applications like surface smoothing, mean-curvature flow, inverse rendering and user-defined editing on implicit geometry.
  arXiv  Detail & Related papers  (2022-04-14T17:59:39Z)
• 3D Equivariant Graph Implicit Functions [51.5559264447605]
  We introduce a novel family of graph implicit functions with equivariant layers that facilitates modeling fine local details. Our method improves over the existing rotation-equivariant implicit function from 0.69 to 0.89 on the ShapeNet reconstruction task.
  arXiv  Detail & Related papers  (2022-03-31T16:51:25Z)
• Tensor Component Analysis for Interpreting the Latent Space of GANs [41.020230946351816]
  This paper addresses the problem of finding interpretable directions in the latent space of pre-trained Generative Adversarial Networks (GANs) Our scheme allows for both linear edits corresponding to the individual modes of the tensor, and non-linear ones that model the multiplicative interactions between them. We show experimentally that we can utilise the former to better separate style- from geometry-based transformations, and the latter to generate an extended set of possible transformations.
  arXiv  Detail & Related papers  (2021-11-23T09:14:39Z)
• Disentangling Geometric Deformation Spaces in Generative Latent Shape Models [5.582957809895198]
  A complete representation of 3D objects requires characterizing the space of deformations in an interpretable manner. We improve on a prior generative model of disentanglement for 3D shapes, wherein the space of object geometry is factorized into rigid orientation, non-rigid pose, and intrinsic shape. The resulting model can be trained from raw 3D shapes, without correspondences, labels, or even rigid alignment.
  arXiv  Detail & Related papers  (2021-02-27T06:54:31Z)
• ResNet-LDDMM: Advancing the LDDMM Framework Using Deep Residual Networks [86.37110868126548]
  In this work, we make use of deep residual neural networks to solve the non-stationary ODE (flow equation) based on a Euler's discretization scheme. We illustrate these ideas on diverse registration problems of 3D shapes under complex topology-preserving transformations.
  arXiv  Detail & Related papers  (2021-02-16T04:07:13Z)
• Understanding Graph Neural Networks with Generalized Geometric Scattering Transforms [67.88675386638043]
  The scattering transform is a multilayered wavelet-based deep learning architecture that acts as a model of convolutional neural networks. We introduce windowed and non-windowed geometric scattering transforms for graphs based upon a very general class of asymmetric wavelets. We show that these asymmetric graph scattering transforms have many of the same theoretical guarantees as their symmetric counterparts.
  arXiv  Detail & Related papers  (2019-11-14T17:23:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.