HeadCraft: Modeling High-Detail Shape Variations for Animated 3DMMs

• URL: http://arxiv.org/abs/2312.14140v1
• Date: Thu, 21 Dec 2023 18:57:52 GMT
• Title: HeadCraft: Modeling High-Detail Shape Variations for Animated 3DMMs
• Authors: Artem Sevastopolsky, Philip-William Grassal, Simon Giebenhain, ShahRukh Athar, Luisa Verdoliva, Matthias Niessner
• Abstract summary: We introduce a generative model for detailed 3D head meshes on top of an articulated 3DMM. We train a StyleGAN model in order to generalize over the UV maps of displacements. We demonstrate the results of unconditional generation and fitting to the full or partial observation.
• Score: 9.790185628415301
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current advances in human head modeling allow to generate plausible-looking 3D head models via neural representations. Nevertheless, constructing complete high-fidelity head models with explicitly controlled animation remains an issue. Furthermore, completing the head geometry based on a partial observation, e.g. coming from a depth sensor, while preserving details is often problematic for the existing methods. We introduce a generative model for detailed 3D head meshes on top of an articulated 3DMM which allows explicit animation and high-detail preservation at the same time. Our method is trained in two stages. First, we register a parametric head model with vertex displacements to each mesh of the recently introduced NPHM dataset of accurate 3D head scans. The estimated displacements are baked into a hand-crafted UV layout. Second, we train a StyleGAN model in order to generalize over the UV maps of displacements. The decomposition of the parametric model and high-quality vertex displacements allows us to animate the model and modify it semantically. We demonstrate the results of unconditional generation and fitting to the full or partial observation. The project page is available at https://seva100.github.io/headcraft.

Related papers

• Towards Native Generative Model for 3D Head Avatar [20.770534728078623]
  We show how to learn a native generative model for 360$circ$ full head from a limited 3D head dataset. Specifically, three major problems are studied: how to effectively utilize various representations for generating the 360$circ$-renderable human head. We hope the proposed models and artist-designed dataset can inspire future research on learning native generative 3D head models from limited 3D datasets.
  arXiv  Detail & Related papers  (2024-10-02T04:04:10Z)
• Head360: Learning a Parametric 3D Full-Head for Free-View Synthesis in 360° [25.86740659962933]
  We build a dataset of artist-designed high-fidelity human heads and propose to create a novel parametric head model from it. Our scheme decouples the facial motion/shape and facial appearance, which are represented by a classic parametric 3D mesh model and an attached neural texture. Experiments show that facial motions and appearances are well disentangled in the parametric space, leading to SOTA performance in rendering and animating quality.
  arXiv  Detail & Related papers  (2024-08-01T05:46:06Z)
• GPHM: Gaussian Parametric Head Model for Monocular Head Avatar Reconstruction [47.113910048252805]
  High-fidelity 3D human head avatars are crucial for applications in VR/AR, digital human, and film production. Recent advances have leveraged morphable face models to generate animated head avatars, representing varying identities and expressions. We introduce 3D Gaussian Parametric Head Model, which employs 3D Gaussians to accurately represent the complexities of the human head.
  arXiv  Detail & Related papers  (2024-07-21T06:03:11Z)
• SuperGaussian: Repurposing Video Models for 3D Super Resolution [67.19266415499139]
  We present a simple, modular, and generic method that upsamples coarse 3D models by adding geometric and appearance details. We demonstrate that it is possible to directly repurpose existing (pretrained) video models for 3D super-resolution.
  arXiv  Detail & Related papers  (2024-06-02T03:44:50Z)
• Probing the 3D Awareness of Visual Foundation Models [56.68380136809413]
  We analyze the 3D awareness of visual foundation models. We conduct experiments using task-specific probes and zero-shot inference procedures on frozen features.
  arXiv  Detail & Related papers  (2024-04-12T17:58:04Z)
• Articulated 3D Head Avatar Generation using Text-to-Image Diffusion Models [107.84324544272481]
  The ability to generate diverse 3D articulated head avatars is vital to a plethora of applications, including augmented reality, cinematography, and education. Recent work on text-guided 3D object generation has shown great promise in addressing these needs. We show that our diffusion-based articulated head avatars outperform state-of-the-art approaches for this task.
  arXiv  Detail & Related papers  (2023-07-10T19:15:32Z)
• MoDA: Modeling Deformable 3D Objects from Casual Videos [84.29654142118018]
  We propose neural dual quaternion blend skinning (NeuDBS) to achieve 3D point deformation without skin-collapsing artifacts. In the endeavor to register 2D pixels across different frames, we establish a correspondence between canonical feature embeddings that encodes 3D points within the canonical space. Our approach can reconstruct 3D models for humans and animals with better qualitative and quantitative performance than state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-04-17T13:49:04Z)
• i3DMM: Deep Implicit 3D Morphable Model of Human Heads [115.19943330455887]
  We present the first deep implicit 3D morphable model (i3DMM) of full heads. It not only captures identity-specific geometry, texture, and expressions of the frontal face, but also models the entire head, including hair. We show the merits of i3DMM using ablation studies, comparisons to state-of-the-art models, and applications such as semantic head editing and texture transfer.
  arXiv  Detail & Related papers  (2020-11-28T15:01:53Z)
• Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction [123.62341095156611]
  Implicit functions represented as deep learning approximations are powerful for reconstructing 3D surfaces. Such features are essential in building flexible models for both computer graphics and computer vision. We present methodology that combines detail-rich implicit functions and parametric representations.
  arXiv  Detail & Related papers  (2020-07-22T13:46:14Z)
• Real-time Simultaneous 3D Head Modeling and Facial Motion Capture with an RGB-D camera [2.3260877354419254]
  We propose a method to build in real-time animated 3D head models using a consumer-grade RGB-D camera. Anyone's head can be instantly reconstructed and his facial motion captured without requiring any training or pre-scanning.
  arXiv  Detail & Related papers  (2020-04-22T13:22:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.