Toward Mesh-Invariant 3D Generative Deep Learning with Geometric Measures

• URL: http://arxiv.org/abs/2306.15762v1
• Date: Tue, 27 Jun 2023 19:27:15 GMT
• Title: Toward Mesh-Invariant 3D Generative Deep Learning with Geometric Measures
• Authors: Thomas Besnier, Sylvain Arguill\`ere, Emery Pierson, Mohamed Daoudi
• Abstract summary: 3D generative modeling is accelerating as the technology allowing the capture of geometric data is developing. Many generative learning algorithms require correspondence between each point when comparing the predicted shape and the target shape. We propose an architecture able to cope with different parameterizations, even during the training phase.
• Score: 2.167843405313757
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: 3D generative modeling is accelerating as the technology allowing the capture of geometric data is developing. However, the acquired data is often inconsistent, resulting in unregistered meshes or point clouds. Many generative learning algorithms require correspondence between each point when comparing the predicted shape and the target shape. We propose an architecture able to cope with different parameterizations, even during the training phase. In particular, our loss function is built upon a kernel-based metric over a representation of meshes using geometric measures such as currents and varifolds. The latter allows to implement an efficient dissimilarity measure with many desirable properties such as robustness to resampling of the mesh or point cloud. We demonstrate the efficiency and resilience of our model with a generative learning task of human faces.

Related papers

• Masked Generative Extractor for Synergistic Representation and 3D Generation of Point Clouds [6.69660410213287]
  We propose an innovative framework called Point-MGE to explore the benefits of deeply integrating 3D representation learning and generative learning. In shape classification, Point-MGE achieved an accuracy of 94.2% (+1.0%) on the ModelNet40 dataset and 92.9% (+5.5%) on the ScanObjectNN dataset. Experimental results also confirmed that Point-MGE can generate high-quality 3D shapes in both unconditional and conditional settings.
  arXiv  Detail & Related papers  (2024-06-25T07:57:03Z)
• Robust 3D Tracking with Quality-Aware Shape Completion [67.9748164949519]
  We propose a synthetic target representation composed of dense and complete point clouds depicting the target shape precisely by shape completion for robust 3D tracking. Specifically, we design a voxelized 3D tracking framework with shape completion, in which we propose a quality-aware shape completion mechanism to alleviate the adverse effect of noisy historical predictions.
  arXiv  Detail & Related papers  (2023-12-17T04:50:24Z)
• Automatic Parameterization for Aerodynamic Shape Optimization via Deep Geometric Learning [60.69217130006758]
  We propose two deep learning models that fully automate shape parameterization for aerodynamic shape optimization. Both models are optimized to parameterize via deep geometric learning to embed human prior knowledge into learned geometric patterns. We perform shape optimization experiments on 2D airfoils and discuss the applicable scenarios for the two models.
  arXiv  Detail & Related papers  (2023-05-03T13:45:40Z)
• GNPM: Geometric-Aware Neural Parametric Models [6.620111952225635]
  We propose a learned parametric model that takes into account the local structure of data to learn disentangled shape and pose latent spaces of 4D dynamics. We evaluate GNPMs on various datasets of humans, and show that it achieves comparable performance to state-of-the-art methods that require dense correspondences during training.
  arXiv  Detail & Related papers  (2022-09-21T19:23:31Z)
• Geometry-Contrastive Transformer for Generalized 3D Pose Transfer [95.56457218144983]
  The intuition of this work is to perceive the geometric inconsistency between the given meshes with the powerful self-attention mechanism. We propose a novel geometry-contrastive Transformer that has an efficient 3D structured perceiving ability to the global geometric inconsistencies. We present a latent isometric regularization module together with a novel semi-synthesized dataset for the cross-dataset 3D pose transfer task.
  arXiv  Detail & Related papers  (2021-12-14T13:14:24Z)
• Scene Synthesis via Uncertainty-Driven Attribute Synchronization [52.31834816911887]
  This paper introduces a novel neural scene synthesis approach that can capture diverse feature patterns of 3D scenes. Our method combines the strength of both neural network-based and conventional scene synthesis approaches.
  arXiv  Detail & Related papers  (2021-08-30T19:45:07Z)
• GLASS: Geometric Latent Augmentation for Shape Spaces [28.533018136138825]
  We use geometrically motivated energies to augment and thus boost a sparse collection of example (training) models. We analyze the Hessian of the as-rigid-as-possible (ARAP) energy to sample from and project to the underlying (local) shape space. We present multiple examples of interesting and meaningful shape variations even when starting from as few as 3-10 training shapes.
  arXiv  Detail & Related papers  (2021-08-06T17:56:23Z)
• DEF: Deep Estimation of Sharp Geometric Features in 3D Shapes [43.853000396885626]
  We propose a learning-based framework for predicting sharp geometric features in sampled 3D shapes. By fusing the result of individual patches, we can process large 3D models, which are impossible to process for existing data-driven methods.
  arXiv  Detail & Related papers  (2020-11-30T18:21:00Z)
• Primal-Dual Mesh Convolutional Neural Networks [62.165239866312334]
  We propose a primal-dual framework drawn from the graph-neural-network literature to triangle meshes. Our method takes features for both edges and faces of a 3D mesh as input and dynamically aggregates them. We provide theoretical insights of our approach using tools from the mesh-simplification literature.
  arXiv  Detail & Related papers  (2020-10-23T14:49:02Z)
• Pix2Surf: Learning Parametric 3D Surface Models of Objects from Images [64.53227129573293]
  We investigate the problem of learning to generate 3D parametric surface representations for novel object instances, as seen from one or more views. We design neural networks capable of generating high-quality parametric 3D surfaces which are consistent between views. Our method is supervised and trained on a public dataset of shapes from common object categories.
  arXiv  Detail & Related papers  (2020-08-18T06:33:40Z)
• PolyGen: An Autoregressive Generative Model of 3D Meshes [22.860421649320287]
  We present an approach which models the mesh directly using a Transformer-based architecture. Our model can condition on a range of inputs, including object classes, voxels, and images. We show that the model is capable of producing high-quality, usable meshes, and establish log-likelihood benchmarks for the mesh-modelling task.
  arXiv  Detail & Related papers  (2020-02-23T17:16:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.