Generalizable Face Landmarking Guided by Conditional Face Warping

• URL: http://arxiv.org/abs/2404.12322v2
• Date: Sun, 21 Apr 2024 08:37:09 GMT
• Title: Generalizable Face Landmarking Guided by Conditional Face Warping
• Authors: Jiayi Liang, Haotian Liu, Hongteng Xu, Dixin Luo,
• Abstract summary: We learn a generalizable face landmarker based on labeled real human faces and unlabeled stylized faces. Our method outperforms existing state-of-the-art domain adaptation methods in face landmarking tasks.
• Score: 34.49985314656207
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As a significant step for human face modeling, editing, and generation, face landmarking aims at extracting facial keypoints from images. A generalizable face landmarker is required in practice because real-world facial images, e.g., the avatars in animations and games, are often stylized in various ways. However, achieving generalizable face landmarking is challenging due to the diversity of facial styles and the scarcity of labeled stylized faces. In this study, we propose a simple but effective paradigm to learn a generalizable face landmarker based on labeled real human faces and unlabeled stylized faces. Our method learns the face landmarker as the key module of a conditional face warper. Given a pair of real and stylized facial images, the conditional face warper predicts a warping field from the real face to the stylized one, in which the face landmarker predicts the ending points of the warping field and provides us with high-quality pseudo landmarks for the corresponding stylized facial images. Applying an alternating optimization strategy, we learn the face landmarker to minimize $i)$ the discrepancy between the stylized faces and the warped real ones and $ii)$ the prediction errors of both real and pseudo landmarks. Experiments on various datasets show that our method outperforms existing state-of-the-art domain adaptation methods in face landmarking tasks, leading to a face landmarker with better generalizability. Code is available at https://plustwo0.github.io/project-face-landmarker.

Related papers

• GaussianHeads: End-to-End Learning of Drivable Gaussian Head Avatars from Coarse-to-fine Representations [54.94362657501809]
  We propose a new method to generate highly dynamic and deformable human head avatars from multi-view imagery in real-time. At the core of our method is a hierarchical representation of head models that allows to capture the complex dynamics of facial expressions and head movements. We train this coarse-to-fine facial avatar model along with the head pose as a learnable parameter in an end-to-end framework.
  arXiv  Detail & Related papers  (2024-09-18T13:05:43Z)
• FlashFace: Human Image Personalization with High-fidelity Identity Preservation [59.76645602354481]
  FlashFace allows users to easily personalize their own photos by providing one or a few reference face images and a text prompt. Our approach is distinguishable from existing human photo customization methods by higher-fidelity identity preservation and better instruction following.
  arXiv  Detail & Related papers  (2024-03-25T17:59:57Z)
• LAFS: Landmark-based Facial Self-supervised Learning for Face Recognition [37.4550614524874]
  We focus on learning facial representations that can be adapted to train effective face recognition models. We explore the learning strategy of unlabeled facial images through self-supervised pretraining. Our method achieves significant improvement over the state-of-the-art on multiple face recognition benchmarks.
  arXiv  Detail & Related papers  (2024-03-13T01:07:55Z)
• A survey and classification of face alignment methods based on face models [0.0]
  We provide a comprehensive analysis of different face models used for face alignment. We include the interpretation and training of the face models along with the examples of fitting the face model to a new face image. We found that 3D-based face models are preferred in cases of extreme face pose, whereas deep learning-based methods often use heatmaps.
  arXiv  Detail & Related papers  (2023-11-06T13:09:04Z)
• Controlling Memorability of Face Images [5.000272778136267]
  We propose a fast approach to modify and control the memorability of face images. We first found a hyperplane in the latent space of StyleGAN to separate high and low memorable images. We analyzed how different layers of the StyleGAN augmented latent space contribute to face memorability.
  arXiv  Detail & Related papers  (2022-02-24T04:33:55Z)
• Pro-UIGAN: Progressive Face Hallucination from Occluded Thumbnails [53.080403912727604]
  We propose a multi-stage Progressive Upsampling and Inpainting Generative Adversarial Network, dubbed Pro-UIGAN. It exploits facial geometry priors to replenish and upsample (8*) the occluded and tiny faces. Pro-UIGAN achieves visually pleasing HR faces, reaching superior performance in downstream tasks.
  arXiv  Detail & Related papers  (2021-08-02T02:29:24Z)
• HifiFace: 3D Shape and Semantic Prior Guided High Fidelity Face Swapping [116.1022638063613]
  We propose HifiFace, which can preserve the face shape of the source face and generate photo-realistic results. We introduce the Semantic Facial Fusion module to optimize the combination of encoder and decoder features.
  arXiv  Detail & Related papers  (2021-06-18T07:39:09Z)
• LandmarkGAN: Synthesizing Faces from Landmarks [43.53204737135101]
  We describe a new method, namely LandmarkGAN, to synthesize faces based on facial landmarks as input. Our method is able to transform a set of facial landmarks into new faces of different subjects, while retains the same facial expression and orientation.
  arXiv  Detail & Related papers  (2020-10-31T13:27:21Z)
• FaR-GAN for One-Shot Face Reenactment [20.894596219099164]
  We present a one-shot face reenactment model, FaR-GAN, that takes only one face image of any given source identity and a target expression as input. The proposed method makes no assumptions about the source identity, facial expression, head pose, or even image background.
  arXiv  Detail & Related papers  (2020-05-13T16:15:37Z)
• Face Hallucination with Finishing Touches [65.14864257585835]
  We present a novel Vivid Face Hallucination Generative Adversarial Network (VividGAN) for simultaneously super-resolving and frontalizing tiny non-frontal face images. VividGAN consists of coarse-level and fine-level Face Hallucination Networks (FHnet) and two discriminators, i.e., Coarse-D and Fine-D. Experiments demonstrate that our VividGAN achieves photo-realistic frontal HR faces, reaching superior performance in downstream tasks.
  arXiv  Detail & Related papers  (2020-02-09T07:33:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.