HeadGAN: One-shot Neural Head Synthesis and Editing

• URL: http://arxiv.org/abs/2012.08261v2
• Date: Mon, 29 Mar 2021 15:19:28 GMT
• Title: HeadGAN: One-shot Neural Head Synthesis and Editing
• Authors: Michail Christos Doukas, Stefanos Zafeiriou, Viktoriia Sharmanska
• Abstract summary: HeadGAN is a system that synthesises on 3D face representations and adapted to the facial geometry of any reference image. The 3D face representation enables HeadGAN to be further used as an efficient method for compression and reconstruction and a tool for expression and pose editing.
• Score: 70.30831163311296
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Recent attempts to solve the problem of head reenactment using a single reference image have shown promising results. However, most of them either perform poorly in terms of photo-realism, or fail to meet the identity preservation problem, or do not fully transfer the driving pose and expression. We propose HeadGAN, a novel system that conditions synthesis on 3D face representations, which can be extracted from any driving video and adapted to the facial geometry of any reference image, disentangling identity from expression. We further improve mouth movements, by utilising audio features as a complementary input. The 3D face representation enables HeadGAN to be further used as an efficient method for compression and reconstruction and a tool for expression and pose editing.

Related papers

• GaussianHeads: End-to-End Learning of Drivable Gaussian Head Avatars from Coarse-to-fine Representations [54.94362657501809]
  We propose a new method to generate highly dynamic and deformable human head avatars from multi-view imagery in real-time. At the core of our method is a hierarchical representation of head models that allows to capture the complex dynamics of facial expressions and head movements. We train this coarse-to-fine facial avatar model along with the head pose as a learnable parameter in an end-to-end framework.
  arXiv  Detail & Related papers  (2024-09-18T13:05:43Z)
• VOODOO XP: Expressive One-Shot Head Reenactment for VR Telepresence [14.010324388059866]
  VOODOO XP is a 3D-aware one-shot head reenactment method that can generate highly expressive facial expressions from any input driver video and a single 2D portrait. We show our solution on a monocular video setting and an end-to-end VR telepresence system for two-way communication.
  arXiv  Detail & Related papers  (2024-05-25T12:33:40Z)
• Parametric Implicit Face Representation for Audio-Driven Facial Reenactment [52.33618333954383]
  We propose a novel audio-driven facial reenactment framework that is both controllable and can generate high-quality talking heads. Specifically, our parametric implicit representation parameterizes the implicit representation with interpretable parameters of 3D face models. Our method can generate more realistic results than previous methods with greater fidelity to the identities and talking styles of speakers.
  arXiv  Detail & Related papers  (2023-06-13T07:08:22Z)
• One-Shot High-Fidelity Talking-Head Synthesis with Deformable Neural Radiance Field [81.07651217942679]
  Talking head generation aims to generate faces that maintain the identity information of the source image and imitate the motion of the driving image. We propose HiDe-NeRF, which achieves high-fidelity and free-view talking-head synthesis.
  arXiv  Detail & Related papers  (2023-04-11T09:47:35Z)
• Free-HeadGAN: Neural Talking Head Synthesis with Explicit Gaze Control [54.079327030892244]
  Free-HeadGAN is a person-generic neural talking head synthesis system. We show that modeling faces with sparse 3D facial landmarks are sufficient for achieving state-of-the-art generative performance.
  arXiv  Detail & Related papers  (2022-08-03T16:46:08Z)
• One-Shot Face Reenactment on Megapixels [10.93616643559957]
  We present a one-shot and high-resolution face reenactment method called MegaFR. We leverage StyleGAN by using 3DMM-based rendering images and overcome the lack of high-quality video datasets. We apply MegaFR to various applications such as face frontalization, eye in-painting, and talking head generation.
  arXiv  Detail & Related papers  (2022-05-26T13:52:04Z)
• Head2HeadFS: Video-based Head Reenactment with Few-shot Learning [64.46913473391274]
  Head reenactment is a challenging task, which aims at transferring the entire head pose from a source person to a target. We propose head2headFS, a novel easily adaptable pipeline for head reenactment. Our video-based rendering network is fine-tuned under a few-shot learning strategy, using only a few samples.
  arXiv  Detail & Related papers  (2021-03-30T10:19:41Z)
• Head2Head++: Deep Facial Attributes Re-Targeting [6.230979482947681]
  We leverage the 3D geometry of faces and Generative Adversarial Networks (GANs) to design a novel deep learning architecture for the task of facial and head reenactment. We manage to capture the complex non-rigid facial motion from the driving monocular performances and synthesise temporally consistent videos. Our system performs end-to-end reenactment in nearly real-time speed (18 fps)
  arXiv  Detail & Related papers  (2020-06-17T23:38:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.