Thinking the Fusion Strategy of Multi-reference Face Reenactment
- URL: http://arxiv.org/abs/2202.10758v1
- Date: Tue, 22 Feb 2022 09:17:26 GMT
- Title: Thinking the Fusion Strategy of Multi-reference Face Reenactment
- Authors: Takuya Yashima, Takuya Narihira, Tamaki Kojima
- Abstract summary: We show that simple extension by using multiple reference images significantly improves generation quality.
We show this by 1) conducting the reconstruction task on publicly available dataset, 2) conducting facial motion transfer on our original dataset which consists of multi-person's head movement video sequences, and 3) using a newly proposed evaluation metric to validate that our method achieves better quantitative results.
- Score: 4.1509697008011175
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: In recent advances of deep generative models, face reenactment -manipulating
and controlling human face, including their head movement-has drawn much
attention for its wide range of applicability. Despite its strong
expressiveness, it is inevitable that the models fail to reconstruct or
accurately generate unseen side of the face of a given single reference image.
Most of existing methods alleviate this problem by learning appearances of
human faces from large amount of data and generate realistic texture at
inference time. Rather than completely relying on what generative models learn,
we show that simple extension by using multiple reference images significantly
improves generation quality. We show this by 1) conducting the reconstruction
task on publicly available dataset, 2) conducting facial motion transfer on our
original dataset which consists of multi-person's head movement video
sequences, and 3) using a newly proposed evaluation metric to validate that our
method achieves better quantitative results.
Related papers
- Active Generation for Image Classification [50.18107721267218]
We propose to address the efficiency of image generation by focusing on the specific needs and characteristics of the model.
With a central tenet of active learning, our method, named ActGen, takes a training-aware approach to image generation.
arXiv Detail & Related papers (2024-03-11T08:45:31Z) - Effective Adapter for Face Recognition in the Wild [72.75516495170199]
We tackle the challenge of face recognition in the wild, where images often suffer from low quality and real-world distortions.
Traditional approaches-either training models directly on degraded images or their enhanced counterparts using face restoration techniques-have proven ineffective.
We propose an effective adapter for augmenting existing face recognition models trained on high-quality facial datasets.
arXiv Detail & Related papers (2023-12-04T08:55:46Z) - HyperReenact: One-Shot Reenactment via Jointly Learning to Refine and
Retarget Faces [47.27033282706179]
We present our method for neural face reenactment, called HyperReenact, that aims to generate realistic talking head images of a source identity.
Our method operates under the one-shot setting (i.e., using a single source frame) and allows for cross-subject reenactment, without requiring subject-specific fine-tuning.
We compare our method both quantitatively and qualitatively against several state-of-the-art techniques on the standard benchmarks of VoxCeleb1 and VoxCeleb2.
arXiv Detail & Related papers (2023-07-20T11:59:42Z) - A Hierarchical Representation Network for Accurate and Detailed Face
Reconstruction from In-The-Wild Images [15.40230841242637]
We present a novel hierarchical representation network (HRN) to achieve accurate and detailed face reconstruction from a single image.
Our framework can be extended to a multi-view fashion by considering detail consistency of different views.
Our method outperforms the existing methods in both reconstruction accuracy and visual effects.
arXiv Detail & Related papers (2023-02-28T09:24:36Z) - Semantic-aware One-shot Face Re-enactment with Dense Correspondence
Estimation [100.60938767993088]
One-shot face re-enactment is a challenging task due to the identity mismatch between source and driving faces.
This paper proposes to use 3D Morphable Model (3DMM) for explicit facial semantic decomposition and identity disentanglement.
arXiv Detail & Related papers (2022-11-23T03:02:34Z) - Finding Directions in GAN's Latent Space for Neural Face Reenactment [45.67273942952348]
This paper is on face/head reenactment where the goal is to transfer the facial pose (3D head orientation and expression) of a target face to a source face.
We take a different approach, bypassing the training of such networks, by using (fine-tuned) pre-trained GANs.
We show that by embedding real images in the GAN latent space, our method can be successfully used for the reenactment of real-world faces.
arXiv Detail & Related papers (2022-01-31T19:14:03Z) - Everything's Talkin': Pareidolia Face Reenactment [119.49707201178633]
Pareidolia Face Reenactment is defined as animating a static illusory face to move in tandem with a human face in the video.
For the large differences between pareidolia face reenactment and traditional human face reenactment, shape variance and texture variance are introduced.
We propose a novel Parametric Unsupervised Reenactment Algorithm to tackle these two challenges.
arXiv Detail & Related papers (2021-04-07T11:19:13Z) - Weakly-Supervised Multi-Face 3D Reconstruction [45.864415499303405]
We propose an effective end-to-end framework for multi-face 3D reconstruction.
We employ the same global camera model for the reconstructed faces in each image, which makes it possible to recover the relative head positions and orientations in the 3D scene.
arXiv Detail & Related papers (2021-01-06T13:15:21Z) - Learning Complete 3D Morphable Face Models from Images and Videos [88.34033810328201]
We present the first approach to learn complete 3D models of face identity geometry, albedo and expression just from images and videos.
We show that our learned models better generalize and lead to higher quality image-based reconstructions than existing approaches.
arXiv Detail & Related papers (2020-10-04T20:51:23Z) - Rotate-and-Render: Unsupervised Photorealistic Face Rotation from
Single-View Images [47.18219551855583]
We propose a novel unsupervised framework that can synthesize photo-realistic rotated faces.
Our key insight is that rotating faces in the 3D space back and forth, and re-rendering them to the 2D plane can serve as a strong self-supervision.
Our approach has superior synthesis quality as well as identity preservation over the state-of-the-art methods.
arXiv Detail & Related papers (2020-03-18T09:54:46Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.