Head2HeadFS: Video-based Head Reenactment with Few-shot Learning
- URL: http://arxiv.org/abs/2103.16229v1
- Date: Tue, 30 Mar 2021 10:19:41 GMT
- Title: Head2HeadFS: Video-based Head Reenactment with Few-shot Learning
- Authors: Michail Christos Doukas, Mohammad Rami Koujan, Viktoriia Sharmanska,
Stefanos Zafeiriou
- Abstract summary: Head reenactment is a challenging task, which aims at transferring the entire head pose from a source person to a target.
We propose head2headFS, a novel easily adaptable pipeline for head reenactment.
Our video-based rendering network is fine-tuned under a few-shot learning strategy, using only a few samples.
- Score: 64.46913473391274
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Over the past years, a substantial amount of work has been done on the
problem of facial reenactment, with the solutions coming mainly from the
graphics community. Head reenactment is an even more challenging task, which
aims at transferring not only the facial expression, but also the entire head
pose from a source person to a target. Current approaches either train
person-specific systems, or use facial landmarks to model human heads, a
representation that might transfer unwanted identity attributes from the source
to the target. We propose head2headFS, a novel easily adaptable pipeline for
head reenactment. We condition synthesis of the target person on dense 3D face
shape information from the source, which enables high quality expression and
pose transfer. Our video-based rendering network is fine-tuned under a few-shot
learning strategy, using only a few samples. This allows for fast adaptation of
a generic generator trained on a multiple-person dataset, into a
person-specific one.
Related papers
- GaussianHeads: End-to-End Learning of Drivable Gaussian Head Avatars from Coarse-to-fine Representations [54.94362657501809]
We propose a new method to generate highly dynamic and deformable human head avatars from multi-view imagery in real-time.
At the core of our method is a hierarchical representation of head models that allows to capture the complex dynamics of facial expressions and head movements.
We train this coarse-to-fine facial avatar model along with the head pose as a learnable parameter in an end-to-end framework.
arXiv Detail & Related papers (2024-09-18T13:05:43Z) - StyleMask: Disentangling the Style Space of StyleGAN2 for Neural Face
Reenactment [47.27033282706179]
We propose a framework that learns to disentangle the identity characteristics of the face from its pose.
We show that the proposed method produces higher quality results even on extreme pose variations.
arXiv Detail & Related papers (2022-09-27T13:22:35Z) - Copy Motion From One to Another: Fake Motion Video Generation [53.676020148034034]
A compelling application of artificial intelligence is to generate a video of a target person performing arbitrary desired motion.
Current methods typically employ GANs with a L2 loss to assess the authenticity of the generated videos.
We propose a theoretically motivated Gromov-Wasserstein loss that facilitates learning the mapping from a pose to a foreground image.
Our method is able to generate realistic target person videos, faithfully copying complex motions from a source person.
arXiv Detail & Related papers (2022-05-03T08:45:22Z) - HeadGAN: One-shot Neural Head Synthesis and Editing [70.30831163311296]
HeadGAN is a system that synthesises on 3D face representations and adapted to the facial geometry of any reference image.
The 3D face representation enables HeadGAN to be further used as an efficient method for compression and reconstruction and a tool for expression and pose editing.
arXiv Detail & Related papers (2020-12-15T12:51:32Z) - Head2Head++: Deep Facial Attributes Re-Targeting [6.230979482947681]
We leverage the 3D geometry of faces and Generative Adversarial Networks (GANs) to design a novel deep learning architecture for the task of facial and head reenactment.
We manage to capture the complex non-rigid facial motion from the driving monocular performances and synthesise temporally consistent videos.
Our system performs end-to-end reenactment in nearly real-time speed (18 fps)
arXiv Detail & Related papers (2020-06-17T23:38:37Z) - ReenactNet: Real-time Full Head Reenactment [50.32988828989691]
We propose a head-to-head system capable of fully transferring the human head 3D pose, facial expressions and eye gaze from a source to a target actor.
Our system produces high-fidelity, temporally-smooth and photo-realistic synthetic videos faithfully transferring the human time-varying head attributes from the source to the target actor.
arXiv Detail & Related papers (2020-05-22T00:51:38Z) - FaR-GAN for One-Shot Face Reenactment [20.894596219099164]
We present a one-shot face reenactment model, FaR-GAN, that takes only one face image of any given source identity and a target expression as input.
The proposed method makes no assumptions about the source identity, facial expression, head pose, or even image background.
arXiv Detail & Related papers (2020-05-13T16:15:37Z) - Audio-driven Talking Face Video Generation with Learning-based
Personalized Head Pose [67.31838207805573]
We propose a deep neural network model that takes an audio signal A of a source person and a short video V of a target person as input.
We outputs a synthesized high-quality talking face video with personalized head pose.
Our method can generate high-quality talking face videos with more distinguishing head movement effects than state-of-the-art methods.
arXiv Detail & Related papers (2020-02-24T10:02:10Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.