Joint Face Image Restoration and Frontalization for Recognition

• URL: http://arxiv.org/abs/2105.09907v1
• Date: Wed, 12 May 2021 03:52:41 GMT
• Title: Joint Face Image Restoration and Frontalization for Recognition
• Authors: Xiaoguang Tu, Jian Zhao, Qiankun Liu, Wenjie Ai, Guodong Guo, Zhifeng Li, Wei Liu, and Jiashi Feng
• Abstract summary: In real-world scenarios, many factors may harm face recognition performance, e.g., large pose, bad illumination,low resolution, blur and noise. Previous efforts usually first restore the low-quality faces to high-quality ones and then perform face recognition. We propose an Multi-Degradation Face Restoration model to restore frontalized high-quality faces from the given low-quality ones.
• Score: 79.78729632975744
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In real-world scenarios, many factors may harm face recognition performance, e.g., large pose, bad illumination,low resolution, blur and noise. To address these challenges, previous efforts usually first restore the low-quality faces to high-quality ones and then perform face recognition. However, most of these methods are stage-wise, which is sub-optimal and deviates from the reality. In this paper, we address all these challenges jointly for unconstrained face recognition. We propose an Multi-Degradation Face Restoration (MDFR) model to restore frontalized high-quality faces from the given low-quality ones under arbitrary facial poses, with three distinct novelties. First, MDFR is a well-designed encoder-decoder architecture which extracts feature representation from an input face image with arbitrary low-quality factors and restores it to a high-quality counterpart. Second, MDFR introduces a pose residual learning strategy along with a 3D-based Pose Normalization Module (PNM), which can perceive the pose gap between the input initial pose and its real-frontal pose to guide the face frontalization. Finally, MDFR can generate frontalized high-quality face images by a single unified network, showing a strong capability of preserving face identity. Qualitative and quantitative experiments on both controlled and in-the-wild benchmarks demonstrate the superiority of MDFR over state-of-the-art methods on both face frontalization and face restoration.

Related papers

• AuthFace: Towards Authentic Blind Face Restoration with Face-oriented Generative Diffusion Prior [13.27748226506837]
  Blind face restoration (BFR) is a fundamental and challenging problem in computer vision. Recent research endeavors rely on facial image priors from the powerful pretrained text-to-image (T2I) diffusion models. We propose AuthFace, which achieves highly authentic face restoration results by exploring a face-oriented generative diffusion prior.
  arXiv  Detail & Related papers  (2024-10-13T14:56:13Z)
• Effective Adapter for Face Recognition in the Wild [72.75516495170199]
  We tackle the challenge of face recognition in the wild, where images often suffer from low quality and real-world distortions. Traditional approaches-either training models directly on degraded images or their enhanced counterparts using face restoration techniques-have proven ineffective. We propose an effective adapter for augmenting existing face recognition models trained on high-quality facial datasets.
  arXiv  Detail & Related papers  (2023-12-04T08:55:46Z)
• A Hierarchical Representation Network for Accurate and Detailed Face Reconstruction from In-The-Wild Images [15.40230841242637]
  We present a novel hierarchical representation network (HRN) to achieve accurate and detailed face reconstruction from a single image. Our framework can be extended to a multi-view fashion by considering detail consistency of different views. Our method outperforms the existing methods in both reconstruction accuracy and visual effects.
  arXiv  Detail & Related papers  (2023-02-28T09:24:36Z)
• A Survey of Deep Face Restoration: Denoise, Super-Resolution, Deblur, Artifact Removal [177.21001709272144]
  Face Restoration (FR) aims to restore High-Quality (HQ) faces from Low-Quality (LQ) input images. This paper comprehensively surveys recent advances in deep learning techniques for face restoration.
  arXiv  Detail & Related papers  (2022-11-05T07:08:15Z)
• Enhancing Quality of Pose-varied Face Restoration with Local Weak Feature Sensing and GAN Prior [29.17397958948725]
  We propose a well-designed blind face restoration network with generative facial prior. Our model performs superior to the prior art for face restoration and face super-resolution tasks.
  arXiv  Detail & Related papers  (2022-05-28T09:23:48Z)
• GMFIM: A Generative Mask-guided Facial Image Manipulation Model for Privacy Preservation [0.7734726150561088]
  We propose a Generative Mask-guided Face Image Manipulation model based on GANs to apply imperceptible editing to the input face image. Our model can achieve better performance against automated face recognition systems in comparison to the state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-01-10T14:09:14Z)
• Identity-Preserving Pose-Robust Face Hallucination Through Face Subspace Prior [14.353574903736343]
  A novel face super-resolution approach will be introduced, in which the hallucinated face is forced to lie in a subspace spanned by the available training faces. A 3D dictionary alignment scheme is also presented, through which the algorithm becomes capable of dealing with low-resolution faces taken in uncontrolled conditions. In extensive experiments carried out on several well-known face datasets, the proposed algorithm shows remarkable performance by generating detailed and close to ground truth results.
  arXiv  Detail & Related papers  (2021-11-20T17:08:38Z)
• One Shot Face Swapping on Megapixels [65.47443090320955]
  This paper proposes the first Megapixel level method for one shot Face Swapping (or MegaFS for short) Complete face representation, stable training, and limited memory usage are the three novel contributions to the success of our method.
  arXiv  Detail & Related papers  (2021-05-11T10:41:47Z)
• SuperFront: From Low-resolution to High-resolution Frontal Face Synthesis [65.35922024067551]
  We propose a generative adversarial network (GAN) -based model to generate high-quality, identity preserving frontal faces. Specifically, we propose SuperFront-GAN to synthesize a high-resolution (HR), frontal face from one-to-many LR faces with various poses. We integrate a super-resolution side-view module into SF-GAN to preserve identity information and fine details of the side-views in HR space.
  arXiv  Detail & Related papers  (2020-12-07T23:30:28Z)
• HiFaceGAN: Face Renovation via Collaborative Suppression and Replenishment [63.333407973913374]
  "Face Renovation"(FR) is a semantic-guided generation problem. "HiFaceGAN" is a multi-stage framework containing several nested CSR units. experiments on both synthetic and real face images have verified the superior performance of HiFaceGAN.
  arXiv  Detail & Related papers  (2020-05-11T11:33:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.