Skull-to-Face: Anatomy-Guided 3D Facial Reconstruction and Editing

• URL: http://arxiv.org/abs/2403.16207v2
• Date: Sun, 22 Dec 2024 04:04:02 GMT
• Title: Skull-to-Face: Anatomy-Guided 3D Facial Reconstruction and Editing
• Authors: Yongqing Liang, Congyi Zhang, Junli Zhao, Wenping Wang, Xin Li,
• Abstract summary: Deducing the 3D face from a skull is a challenging task in forensic science and archaeology. This paper proposes an end-to-end 3D face reconstruction pipeline and an exploration method. Experiments conducted on a real skull-face dataset demonstrated the effectiveness of our proposed pipeline.
• Score: 34.39385635485985
• License:
• Abstract: Deducing the 3D face from a skull is a challenging task in forensic science and archaeology. This paper proposes an end-to-end 3D face reconstruction pipeline and an exploration method that can conveniently create textured, realistic faces that match the given skull. To this end, we propose a tissue-guided face creation and adaptation scheme. With the help of the state-of-the-art text-to-image diffusion model and parametric face model, we first generate an initial reference 3D face, whose biological profile aligns with the given skull. Then, with the help of tissue thickness distribution, we modify these initial faces to match the skull through a latent optimization process. The joint distribution of tissue thickness is learned on a set of skull landmarks using a collection of scanned skull-face pairs. We also develop an efficient face adaptation tool to allow users to interactively adjust tissue thickness either globally or at local regions to explore different plausible faces. Experiments conducted on a real skull-face dataset demonstrated the effectiveness of our proposed pipeline in terms of reconstruction accuracy, diversity, and stability. Our project page is https://xmlyqing00.github.io/skull-to-face-page.

Related papers

• 3D Face Reconstruction With Geometry Details From a Single Color Image Under Occluded Scenes [4.542616945567623]
  3D face reconstruction technology aims to generate a face stereo model naturally and realistically. Previous deep face reconstruction approaches are typically designed to generate convincing textures. By introducing bump mapping, we successfully added mid-level details to coarse 3D faces.
  arXiv  Detail & Related papers  (2024-12-25T15:16:02Z)
• Generative Face Parsing Map Guided 3D Face Reconstruction Under Occluded Scenes [4.542616945567623]
  A complete face parsing map generation method guided by landmarks is proposed. An excellent anti-occlusion face reconstruction method should ensure the authenticity of the output.
  arXiv  Detail & Related papers  (2024-12-25T14:49:41Z)
• Generating 2D and 3D Master Faces for Dictionary Attacks with a Network-Assisted Latent Space Evolution [68.8204255655161]
  A master face is a face image that passes face-based identity authentication for a high percentage of the population. We optimize these faces for 2D and 3D face verification models. In 3D, we generate faces using the 2D StyleGAN2 generator and predict a 3D structure using a deep 3D face reconstruction network.
  arXiv  Detail & Related papers  (2022-11-25T09:15:38Z)
• SCULPTOR: Skeleton-Consistent Face Creation Using a Learned Parametric Generator [42.25745590793068]
  We present SCULPTOR, 3D face creations with Skeleton Consistency Using a Learned Parametric facial generaTOR. At the core of SCULPTOR is LUCY, the first large-scale shape-skeleton face dataset in collaboration with plastic surgeons.
  arXiv  Detail & Related papers  (2022-09-14T05:21:20Z)
• FaceTuneGAN: Face Autoencoder for Convolutional Expression Transfer Using Neural Generative Adversarial Networks [0.7043489166804575]
  We present FaceTuneGAN, a new 3D face model representation decomposing and encoding separately facial identity and facial expression. We propose a first adaptation of image-to-image translation networks, that have successfully been used in the 2D domain, to 3D face geometry.
  arXiv  Detail & Related papers  (2021-12-01T14:42:03Z)
• Detailed Avatar Recovery from Single Image [50.82102098057822]
  This paper presents a novel framework to recover emphdetailed avatar from a single image. We use the deep neural networks to refine the 3D shape in a Hierarchical Mesh Deformation framework. Our method can restore detailed human body shapes with complete textures beyond skinned models.
  arXiv  Detail & Related papers  (2021-08-06T03:51:26Z)
• HifiFace: 3D Shape and Semantic Prior Guided High Fidelity Face Swapping [116.1022638063613]
  We propose HifiFace, which can preserve the face shape of the source face and generate photo-realistic results. We introduce the Semantic Facial Fusion module to optimize the combination of encoder and decoder features.
  arXiv  Detail & Related papers  (2021-06-18T07:39:09Z)
• Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstruction [76.1612334630256]
  We harness the power of Generative Adversarial Networks (GANs) and Deep Convolutional Neural Networks (DCNNs) to reconstruct the facial texture and shape from single images. We demonstrate excellent results in photorealistic and identity preserving 3D face reconstructions and achieve for the first time, facial texture reconstruction with high-frequency details.
  arXiv  Detail & Related papers  (2021-05-16T16:35:44Z)
• Face Super-Resolution Guided by 3D Facial Priors [92.23902886737832]
  We propose a novel face super-resolution method that explicitly incorporates 3D facial priors which grasp the sharp facial structures. Our work is the first to explore 3D morphable knowledge based on the fusion of parametric descriptions of face attributes. The proposed 3D priors achieve superior face super-resolution results over the state-of-the-arts.
  arXiv  Detail & Related papers  (2020-07-18T15:26:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.