Related papers: Towards Real-World Blind Face Restoration with Generative Facial Prior

Towards Real-World Blind Face Restoration with Generative Facial Prior

URL: http://arxiv.org/abs/2101.04061v1
Date: Mon, 11 Jan 2021 17:54:38 GMT
Title: Towards Real-World Blind Face Restoration with Generative Facial Prior
Authors: Xintao Wang, Yu Li, Honglun Zhang, Ying Shan
Abstract summary: Blind face restoration usually relies on facial priors, such as facial geometry prior or reference prior, to restore realistic and faithful details. We propose GFP-GAN that leverages rich and diverse priors encapsulated in a pretrained face GAN for blind face restoration. Our method achieves superior performance to prior art on both synthetic and real-world datasets.
Score: 19.080349401153097
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: Blind face restoration usually relies on facial priors, such as facial geometry prior or reference prior, to restore realistic and faithful details. However, very low-quality inputs cannot offer accurate geometric prior while high-quality references are inaccessible, limiting the applicability in real-world scenarios. In this work, we propose GFP-GAN that leverages rich and diverse priors encapsulated in a pretrained face GAN for blind face restoration. This Generative Facial Prior (GFP) is incorporated into the face restoration process via novel channel-split spatial feature transform layers, which allow our method to achieve a good balance of realness and fidelity. Thanks to the powerful generative facial prior and delicate designs, our GFP-GAN could jointly restore facial details and enhance colors with just a single forward pass, while GAN inversion methods require expensive image-specific optimization at inference. Extensive experiments show that our method achieves superior performance to prior art on both synthetic and real-world datasets.

Related papers

Towards Real-World Blind Face Restoration with Generative Diffusion Prior [69.84480964328465]
Blind face restoration is an important task in computer vision and has gained significant attention due to its wide-range applications. We propose BFRffusion which is thoughtfully designed to effectively extract features from low-quality face images. We also build a privacy-preserving face dataset called PFHQ with balanced attributes like race, gender, and age.
arXiv Detail & Related papers (2023-12-25T14:16:24Z)
FaceFormer: Scale-aware Blind Face Restoration with Transformers [18.514630131883536]
We propose a novel scale-aware blind face restoration framework, named FaceFormer, which formulates facial feature restoration as scale-aware transformation. Our proposed method trained with synthetic dataset generalizes better to a natural low quality images than current state-of-the-arts.
arXiv Detail & Related papers (2022-07-20T10:08:34Z)
Enhancing Quality of Pose-varied Face Restoration with Local Weak Feature Sensing and GAN Prior [29.17397958948725]
We propose a well-designed blind face restoration network with generative facial prior. Our model performs superior to the prior art for face restoration and face super-resolution tasks.
arXiv Detail & Related papers (2022-05-28T09:23:48Z)
TANet: A new Paradigm for Global Face Super-resolution via Transformer-CNN Aggregation Network [72.41798177302175]
We propose a novel paradigm based on the self-attention mechanism (i.e., the core of Transformer) to fully explore the representation capacity of the facial structure feature. Specifically, we design a Transformer-CNN aggregation network (TANet) consisting of two paths, in which one path uses CNNs responsible for restoring fine-grained facial details. By aggregating the features from the above two paths, the consistency of global facial structure and fidelity of local facial detail restoration are strengthened simultaneously.
arXiv Detail & Related papers (2021-09-16T18:15:07Z)
Pro-UIGAN: Progressive Face Hallucination from Occluded Thumbnails [53.080403912727604]
We propose a multi-stage Progressive Upsampling and Inpainting Generative Adversarial Network, dubbed Pro-UIGAN. It exploits facial geometry priors to replenish and upsample (8*) the occluded and tiny faces. Pro-UIGAN achieves visually pleasing HR faces, reaching superior performance in downstream tasks.
arXiv Detail & Related papers (2021-08-02T02:29:24Z)
Joint Face Image Restoration and Frontalization for Recognition [79.78729632975744]
In real-world scenarios, many factors may harm face recognition performance, e.g., large pose, bad illumination,low resolution, blur and noise. Previous efforts usually first restore the low-quality faces to high-quality ones and then perform face recognition. We propose an Multi-Degradation Face Restoration model to restore frontalized high-quality faces from the given low-quality ones.
arXiv Detail & Related papers (2021-05-12T03:52:41Z)
Facial Attribute Transformers for Precise and Robust Makeup Transfer [79.41060385695977]
We propose a novel Facial Attribute Transformer (FAT) and its variant Spatial FAT for high-quality makeup transfer. FAT is able to model the semantic correspondences and interactions between the source face and reference face, and then precisely estimate and transfer the facial attributes. We also integrate thin plate splines (TPS) into FAT, thus creating Spatial FAT, which is the first method that can transfer geometric attributes in addition to color and texture.
arXiv Detail & Related papers (2021-04-07T03:39:02Z)
HiFaceGAN: Face Renovation via Collaborative Suppression and Replenishment [63.333407973913374]
"Face Renovation"(FR) is a semantic-guided generation problem. "HiFaceGAN" is a multi-stage framework containing several nested CSR units. experiments on both synthetic and real face images have verified the superior performance of HiFaceGAN.
arXiv Detail & Related papers (2020-05-11T11:33:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.