Transfer Learning using Spectral Convolutional Autoencoders on Semi-Regular Surface Meshes

• URL: http://arxiv.org/abs/2212.05810v1
• Date: Mon, 12 Dec 2022 10:13:21 GMT
• Title: Transfer Learning using Spectral Convolutional Autoencoders on Semi-Regular Surface Meshes
• Authors: Sara Hahner, Felix Kerkhoff, Jochen Garcke
• Abstract summary: We propose a novel spectral CoSMA (Convolutional Semi-Regular Mesh Autoencoder) network. It reconstructs surfaces not presented during training and generalizes the deformation behavior of the surfaces' patches. Our transfer learning errors on unseen shapes are 40% lower than those from models learned directly on the data.
• Score: 0.966840768820136
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The underlying dynamics and patterns of 3D surface meshes deforming over time can be discovered by unsupervised learning, especially autoencoders, which calculate low-dimensional embeddings of the surfaces. To study the deformation patterns of unseen shapes by transfer learning, we want to train an autoencoder that can analyze new surface meshes without training a new network. Here, most state-of-the-art autoencoders cannot handle meshes of different connectivity and therefore have limited to no generalization capacities to new meshes. Also, reconstruction errors strongly increase in comparison to the errors for the training shapes. To address this, we propose a novel spectral CoSMA (Convolutional Semi-Regular Mesh Autoencoder) network. This patch-based approach is combined with a surface-aware training. It reconstructs surfaces not presented during training and generalizes the deformation behavior of the surfaces' patches. The novel approach reconstructs unseen meshes from different datasets in superior quality compared to state-of-the-art autoencoders that have been trained on these shapes. Our transfer learning errors on unseen shapes are 40% lower than those from models learned directly on the data. Furthermore, baseline autoencoders detect deformation patterns of unseen mesh sequences only for the whole shape. In contrast, due to the employed regional patches and stable reconstruction quality, we can localize where on the surfaces these deformation patterns manifest.

Related papers

• Flatten Anything: Unsupervised Neural Surface Parameterization [76.4422287292541]
  We introduce the Flatten Anything Model (FAM), an unsupervised neural architecture to achieve global free-boundary surface parameterization. Compared with previous methods, our FAM directly operates on discrete surface points without utilizing connectivity information. Our FAM is fully-automated without the need for pre-cutting and can deal with highly-complex topologies.
  arXiv  Detail & Related papers  (2024-05-23T14:39:52Z)
• DynoSurf: Neural Deformation-based Temporally Consistent Dynamic Surface Reconstruction [93.18586302123633]
  This paper explores the problem of reconstructing temporally consistent surfaces from a 3D point cloud sequence without correspondence. We propose DynoSurf, an unsupervised learning framework integrating a template surface representation with a learnable deformation field. Experimental results demonstrate the significant superiority of DynoSurf over current state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-03-18T08:58:48Z)
• Are We Using Autoencoders in a Wrong Way? [3.110260251019273]
  Autoencoders are used for dimensionality reduction, anomaly detection and feature extraction. We revisited the standard training for the undercomplete Autoencoder modifying the shape of the latent space. We also explored the behaviour of the latent space in the case of reconstruction of a random sample from the whole dataset.
  arXiv  Detail & Related papers  (2023-09-04T11:22:43Z)
• Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis [90.26556260531707]
  DMTet is a conditional generative model that can synthesize high-resolution 3D shapes using simple user guides such as coarse voxels. Unlike deep 3D generative models that directly generate explicit representations such as meshes, our model can synthesize shapes with arbitrary topology.
  arXiv  Detail & Related papers  (2021-11-08T05:29:35Z)
• Mesh Convolutional Autoencoder for Semi-Regular Meshes of Different Sizes [0.0]
  State-of-the-art mesh convolutional autoencoders require a fixed connectivity of all input meshes handled by the autoencoder. We transform the discretization of the surfaces to semi-regular meshes that have a locally regular connectivity and whose meshing is hierarchical. We apply the same mesh autoencoder to different datasets and our reconstruction error is more than 50% lower than the error from state-of-the-art models.
  arXiv  Detail & Related papers  (2021-10-18T15:30:40Z)
• Unsupervised Dense Deformation Embedding Network for Template-Free Shape Correspondence [18.48814403488283]
  Current deep learning based methods require the supervision of dense annotations to learn per-point translations. We develop a new Unsupervised Deformation Embedding Network (i.e., UD2E-Net), which learns to predict deformations between non-rigid shapes from dense local features. Our UD2E-Net outperforms state-of-the-art unsupervised methods by 24% on Faust Inter challenge and even supervised methods by 13% on Faust Intra challenge.
  arXiv  Detail & Related papers  (2021-08-26T07:07:19Z)
• Learning Skeletal Articulations with Neural Blend Shapes [57.879030623284216]
  We develop a neural technique for articulating 3D characters using enveloping with a pre-defined skeletal structure. Our framework learns to rig and skin characters with the same articulation structure. We propose neural blend shapes which improve the deformation quality in the joint regions.
  arXiv  Detail & Related papers  (2021-05-06T05:58:13Z)
• Shape My Face: Registering 3D Face Scans by Surface-to-Surface Translation [75.59415852802958]
  Shape-My-Face (SMF) is a powerful encoder-decoder architecture based on an improved point cloud encoder, a novel visual attention mechanism, graph convolutional decoders with skip connections, and a specialized mouth model. Our model provides topologically-sound meshes with minimal supervision, offers faster training time, has orders of magnitude fewer trainable parameters, is more robust to noise, and can generalize to previously unseen datasets.
  arXiv  Detail & Related papers  (2020-12-16T20:02:36Z)
• Neural Subdivision [58.97214948753937]
  This paper introduces Neural Subdivision, a novel framework for data-driven coarseto-fine geometry modeling. We optimize for the same set of network weights across all local mesh patches, thus providing an architecture that is not constrained to a specific input mesh, fixed genus, or category. We demonstrate that even when trained on a single high-resolution mesh our method generates reasonable subdivisions for novel shapes.
  arXiv  Detail & Related papers  (2020-05-04T20:03:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.