HeadCraft: Modeling High-Detail Shape Variations for Animated 3DMMs
- URL: http://arxiv.org/abs/2312.14140v1
- Date: Thu, 21 Dec 2023 18:57:52 GMT
- Title: HeadCraft: Modeling High-Detail Shape Variations for Animated 3DMMs
- Authors: Artem Sevastopolsky, Philip-William Grassal, Simon Giebenhain,
ShahRukh Athar, Luisa Verdoliva, Matthias Niessner
- Abstract summary: We introduce a generative model for detailed 3D head meshes on top of an articulated 3DMM.
We train a StyleGAN model in order to generalize over the UV maps of displacements.
We demonstrate the results of unconditional generation and fitting to the full or partial observation.
- Score: 9.790185628415301
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Current advances in human head modeling allow to generate plausible-looking
3D head models via neural representations. Nevertheless, constructing complete
high-fidelity head models with explicitly controlled animation remains an
issue. Furthermore, completing the head geometry based on a partial
observation, e.g. coming from a depth sensor, while preserving details is often
problematic for the existing methods. We introduce a generative model for
detailed 3D head meshes on top of an articulated 3DMM which allows explicit
animation and high-detail preservation at the same time. Our method is trained
in two stages. First, we register a parametric head model with vertex
displacements to each mesh of the recently introduced NPHM dataset of accurate
3D head scans. The estimated displacements are baked into a hand-crafted UV
layout. Second, we train a StyleGAN model in order to generalize over the UV
maps of displacements. The decomposition of the parametric model and
high-quality vertex displacements allows us to animate the model and modify it
semantically. We demonstrate the results of unconditional generation and
fitting to the full or partial observation. The project page is available at
https://seva100.github.io/headcraft.
Related papers
- SuperGaussian: Repurposing Video Models for 3D Super Resolution [67.19266415499139]
We present a simple, modular, and generic method that upsamples coarse 3D models by adding geometric and appearance details.
We demonstrate that it is possible to directly repurpose existing (pretrained) video models for 3D super-resolution.
arXiv Detail & Related papers (2024-06-02T03:44:50Z) - Probing the 3D Awareness of Visual Foundation Models [56.68380136809413]
We analyze the 3D awareness of visual foundation models.
We conduct experiments using task-specific probes and zero-shot inference procedures on frozen features.
arXiv Detail & Related papers (2024-04-12T17:58:04Z) - HeadGaS: Real-Time Animatable Head Avatars via 3D Gaussian Splatting [9.98045783250373]
We propose HeadGaS, the first model to use 3D Gaussian Splats (3DGS) for 3D head reconstruction and animation.
We demonstrate that HeadGaS delivers state-of-the-art results in real-time inference frame rates, which surpasses baselines by up to 2dB.
arXiv Detail & Related papers (2023-12-05T17:19:22Z) - Articulated 3D Head Avatar Generation using Text-to-Image Diffusion
Models [107.84324544272481]
The ability to generate diverse 3D articulated head avatars is vital to a plethora of applications, including augmented reality, cinematography, and education.
Recent work on text-guided 3D object generation has shown great promise in addressing these needs.
We show that our diffusion-based articulated head avatars outperform state-of-the-art approaches for this task.
arXiv Detail & Related papers (2023-07-10T19:15:32Z) - MoDA: Modeling Deformable 3D Objects from Casual Videos [84.29654142118018]
We propose neural dual quaternion blend skinning (NeuDBS) to achieve 3D point deformation without skin-collapsing artifacts.
In the endeavor to register 2D pixels across different frames, we establish a correspondence between canonical feature embeddings that encodes 3D points within the canonical space.
Our approach can reconstruct 3D models for humans and animals with better qualitative and quantitative performance than state-of-the-art methods.
arXiv Detail & Related papers (2023-04-17T13:49:04Z) - Next3D: Generative Neural Texture Rasterization for 3D-Aware Head
Avatars [36.4402388864691]
3D-aware generative adversarial networks (GANs) synthesize high-fidelity and multi-view-consistent facial images using only collections of single-view 2D imagery.
Recent efforts incorporate 3D Morphable Face Model (3DMM) to describe deformation in generative radiance fields either explicitly or implicitly.
We propose a novel 3D GAN framework for unsupervised learning of generative, high-quality and 3D-consistent facial avatars from unstructured 2D images.
arXiv Detail & Related papers (2022-11-21T06:40:46Z) - PoseBERT: A Generic Transformer Module for Temporal 3D Human Modeling [23.420076136028687]
PoseBERT is a transformer module that is fully trained on 3D Motion Capture data via masked modeling.
It is simple, generic and versatile, as it can be plugged on top of any image-based model to transform it in a video-based model.
Our experimental results validate that adding PoseBERT on top of various state-of-the-art pose estimation methods consistently improves their performances.
arXiv Detail & Related papers (2022-08-22T11:30:14Z) - i3DMM: Deep Implicit 3D Morphable Model of Human Heads [115.19943330455887]
We present the first deep implicit 3D morphable model (i3DMM) of full heads.
It not only captures identity-specific geometry, texture, and expressions of the frontal face, but also models the entire head, including hair.
We show the merits of i3DMM using ablation studies, comparisons to state-of-the-art models, and applications such as semantic head editing and texture transfer.
arXiv Detail & Related papers (2020-11-28T15:01:53Z) - Combining Implicit Function Learning and Parametric Models for 3D Human
Reconstruction [123.62341095156611]
Implicit functions represented as deep learning approximations are powerful for reconstructing 3D surfaces.
Such features are essential in building flexible models for both computer graphics and computer vision.
We present methodology that combines detail-rich implicit functions and parametric representations.
arXiv Detail & Related papers (2020-07-22T13:46:14Z) - Real-time Simultaneous 3D Head Modeling and Facial Motion Capture with
an RGB-D camera [2.3260877354419254]
We propose a method to build in real-time animated 3D head models using a consumer-grade RGB-D camera.
Anyone's head can be instantly reconstructed and his facial motion captured without requiring any training or pre-scanning.
arXiv Detail & Related papers (2020-04-22T13:22:21Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.