Instant recovery of shape from spectrum via latent space connections
- URL: http://arxiv.org/abs/2003.06523v4
- Date: Wed, 4 Nov 2020 21:53:40 GMT
- Title: Instant recovery of shape from spectrum via latent space connections
- Authors: Riccardo Marin, Arianna Rampini, Umberto Castellani, Emanuele
Rodol\`a, Maks Ovsjanikov, Simone Melzi
- Abstract summary: We introduce the first learning-based method for recovering shapes from Laplacian spectra.
Given an auto-encoder, our model takes the form of a cycle-consistent module to map latent vectors to sequences of eigenvalues.
Our data-driven approach replaces the need for ad-hoc regularizers required by prior methods, while providing more accurate results at a fraction of the computational cost.
- Score: 33.83258865005668
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We introduce the first learning-based method for recovering shapes from
Laplacian spectra. Given an auto-encoder, our model takes the form of a
cycle-consistent module to map latent vectors to sequences of eigenvalues. This
module provides an efficient and effective linkage between spectrum and
geometry of a given shape. Our data-driven approach replaces the need for
ad-hoc regularizers required by prior methods, while providing more accurate
results at a fraction of the computational cost. Our learning model applies
without modifications across different dimensions (2D and 3D shapes alike),
representations (meshes, contours and point clouds), as well as across
different shape classes, and admits arbitrary resolution of the input spectrum
without affecting complexity. The increased flexibility allows us to provide a
proxy to differentiable eigendecomposition and to address notoriously difficult
tasks in 3D vision and geometry processing within a unified framework,
including shape generation from spectrum, mesh super-resolution, shape
exploration, style transfer, spectrum estimation from point clouds,
segmentation transfer and point-to-point matching.
Related papers
- Spectral Meets Spatial: Harmonising 3D Shape Matching and Interpolation [50.376243444909136]
We present a unified framework to predict both point-wise correspondences and shape between 3D shapes.
We combine the deep functional map framework with classical surface deformation models to map shapes in both spectral and spatial domains.
arXiv Detail & Related papers (2024-02-29T07:26:23Z) - Geometrically Consistent Partial Shape Matching [50.29468769172704]
Finding correspondences between 3D shapes is a crucial problem in computer vision and graphics.
An often neglected but essential property of matching geometrics is consistency.
We propose a novel integer linear programming partial shape matching formulation.
arXiv Detail & Related papers (2023-09-10T12:21:42Z) - Learning Modulated Transformation in GANs [69.95217723100413]
We equip the generator in generative adversarial networks (GANs) with a plug-and-play module, termed as modulated transformation module (MTM)
MTM predicts spatial offsets under the control of latent codes, based on which the convolution operation can be applied at variable locations.
It is noteworthy that towards human generation on the challenging TaiChi dataset, we improve the FID of StyleGAN3 from 21.36 to 13.60, demonstrating the efficacy of learning modulated geometry transformation.
arXiv Detail & Related papers (2023-08-29T17:51:22Z) - Neural Wavelet-domain Diffusion for 3D Shape Generation, Inversion, and
Manipulation [54.09274684734721]
We present a new approach for 3D shape generation, inversion, and manipulation, through a direct generative modeling on a continuous implicit representation in wavelet domain.
Specifically, we propose a compact wavelet representation with a pair of coarse and detail coefficient volumes to implicitly represent 3D shapes via truncated signed distance functions and multi-scale biorthogonal wavelets.
We may jointly train an encoder network to learn a latent space for inverting shapes, allowing us to enable a rich variety of whole-shape and region-aware shape manipulations.
arXiv Detail & Related papers (2023-02-01T02:47:53Z) - Representing Shape Collections with Alignment-Aware Linear Models [17.635846912560627]
We revisit the classical representation of 3D point clouds as linear shape models.
Our key insight is to leverage deep learning to represent a collection of shapes as affine transformations.
arXiv Detail & Related papers (2021-09-03T16:28:34Z) - Learning to generate shape from global-local spectra [0.0]
We build our method on top of recent advances on the so called shape-from-spectrum paradigm.
We consider the spectrum as a natural and ready to use representation to encode variability of the shapes.
Our results confirm the improvement of the proposed approach in comparison to existing and alternative methods.
arXiv Detail & Related papers (2021-08-04T16:39:56Z) - 3D Shape Registration Using Spectral Graph Embedding and Probabilistic
Matching [24.41451985857662]
We address the problem of 3D shape registration and we propose a novel technique based on spectral graph theory and probabilistic matching.
The main contribution of this chapter is to extend the spectral graph matching methods to very large graphs by combining spectral graph matching with Laplacian embedding.
arXiv Detail & Related papers (2021-06-21T15:02:31Z) - Disentangling Geometric Deformation Spaces in Generative Latent Shape
Models [5.582957809895198]
A complete representation of 3D objects requires characterizing the space of deformations in an interpretable manner.
We improve on a prior generative model of disentanglement for 3D shapes, wherein the space of object geometry is factorized into rigid orientation, non-rigid pose, and intrinsic shape.
The resulting model can be trained from raw 3D shapes, without correspondences, labels, or even rigid alignment.
arXiv Detail & Related papers (2021-02-27T06:54:31Z) - ResNet-LDDMM: Advancing the LDDMM Framework Using Deep Residual Networks [86.37110868126548]
In this work, we make use of deep residual neural networks to solve the non-stationary ODE (flow equation) based on a Euler's discretization scheme.
We illustrate these ideas on diverse registration problems of 3D shapes under complex topology-preserving transformations.
arXiv Detail & Related papers (2021-02-16T04:07:13Z) - Dense Non-Rigid Structure from Motion: A Manifold Viewpoint [162.88686222340962]
Non-Rigid Structure-from-Motion (NRSfM) problem aims to recover 3D geometry of a deforming object from its 2D feature correspondences across multiple frames.
We show that our approach significantly improves accuracy, scalability, and robustness against noise.
arXiv Detail & Related papers (2020-06-15T09:15:54Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.