Pro-UIGAN: Progressive Face Hallucination from Occluded Thumbnails

• URL: http://arxiv.org/abs/2108.00602v3
• Date: Thu, 5 Aug 2021 00:55:43 GMT
• Title: Pro-UIGAN: Progressive Face Hallucination from Occluded Thumbnails
• Authors: Yang Zhang, Xin Yu, Xiaobo Lu, Ping Liu
• Abstract summary: We propose a multi-stage Progressive Upsampling and Inpainting Generative Adversarial Network, dubbed Pro-UIGAN. It exploits facial geometry priors to replenish and upsample (8*) the occluded and tiny faces. Pro-UIGAN achieves visually pleasing HR faces, reaching superior performance in downstream tasks.
• Score: 53.080403912727604
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we study the task of hallucinating an authentic high-resolution (HR) face from an occluded thumbnail. We propose a multi-stage Progressive Upsampling and Inpainting Generative Adversarial Network, dubbed Pro-UIGAN, which exploits facial geometry priors to replenish and upsample (8*) the occluded and tiny faces (16*16 pixels). Pro-UIGAN iteratively (1) estimates facial geometry priors for low-resolution (LR) faces and (2) acquires non-occluded HR face images under the guidance of the estimated priors. Our multi-stage hallucination network super-resolves and inpaints occluded LR faces in a coarse-to-fine manner, thus reducing unwanted blurriness and artifacts significantly. Specifically, we design a novel cross-modal transformer module for facial priors estimation, in which an input face and its landmark features are formulated as queries and keys, respectively. Such a design encourages joint feature learning across the input facial and landmark features, and deep feature correspondences will be discovered by attention. Thus, facial appearance features and facial geometry priors are learned in a mutual promotion manner. Extensive experiments demonstrate that our Pro-UIGAN achieves visually pleasing HR faces, reaching superior performance in downstream tasks, i.e., face alignment, face parsing, face recognition and expression classification, compared with other state-of-the-art (SotA) methods.

Related papers

• A Generalist FaceX via Learning Unified Facial Representation [77.74407008931486]
  FaceX is a novel facial generalist model capable of handling diverse facial tasks simultaneously. Our versatile FaceX achieves competitive performance compared to elaborate task-specific models on popular facial editing tasks.
  arXiv  Detail & Related papers  (2023-12-31T17:41:48Z)
• Enhancing Quality of Pose-varied Face Restoration with Local Weak Feature Sensing and GAN Prior [29.17397958948725]
  We propose a well-designed blind face restoration network with generative facial prior. Our model performs superior to the prior art for face restoration and face super-resolution tasks.
  arXiv  Detail & Related papers  (2022-05-28T09:23:48Z)
• Identity-Preserving Pose-Robust Face Hallucination Through Face Subspace Prior [14.353574903736343]
  A novel face super-resolution approach will be introduced, in which the hallucinated face is forced to lie in a subspace spanned by the available training faces. A 3D dictionary alignment scheme is also presented, through which the algorithm becomes capable of dealing with low-resolution faces taken in uncontrolled conditions. In extensive experiments carried out on several well-known face datasets, the proposed algorithm shows remarkable performance by generating detailed and close to ground truth results.
  arXiv  Detail & Related papers  (2021-11-20T17:08:38Z)
• LR-to-HR Face Hallucination with an Adversarial Progressive Attribute-Induced Network [67.64536397027229]
  Face super-resolution is a challenging and highly ill-posed problem. We propose an end-to-end progressive learning framework incorporating facial attributes. We show that the proposed approach can yield satisfactory face hallucination images outperforming other state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-09-29T19:50:45Z)
• SuperFront: From Low-resolution to High-resolution Frontal Face Synthesis [65.35922024067551]
  We propose a generative adversarial network (GAN) -based model to generate high-quality, identity preserving frontal faces. Specifically, we propose SuperFront-GAN to synthesize a high-resolution (HR), frontal face from one-to-many LR faces with various poses. We integrate a super-resolution side-view module into SF-GAN to preserve identity information and fine details of the side-views in HR space.
  arXiv  Detail & Related papers  (2020-12-07T23:30:28Z)
• Face Hallucination via Split-Attention in Split-Attention Network [58.30436379218425]
  convolutional neural networks (CNNs) have been widely employed to promote the face hallucination. We propose a novel external-internal split attention group (ESAG) to take into account the overall facial profile and fine texture details simultaneously. By fusing the features from these two paths, the consistency of facial structure and the fidelity of facial details are strengthened.
  arXiv  Detail & Related papers  (2020-10-22T10:09:31Z)
• Face Hallucination with Finishing Touches [65.14864257585835]
  We present a novel Vivid Face Hallucination Generative Adversarial Network (VividGAN) for simultaneously super-resolving and frontalizing tiny non-frontal face images. VividGAN consists of coarse-level and fine-level Face Hallucination Networks (FHnet) and two discriminators, i.e., Coarse-D and Fine-D. Experiments demonstrate that our VividGAN achieves photo-realistic frontal HR faces, reaching superior performance in downstream tasks.
  arXiv  Detail & Related papers  (2020-02-09T07:33:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.