W-Net: A Facial Feature-Guided Face Super-Resolution Network

• URL: http://arxiv.org/abs/2406.00676v3
• Date: Sun, 23 Jun 2024 05:46:55 GMT
• Title: W-Net: A Facial Feature-Guided Face Super-Resolution Network
• Authors: Hao Liu, Yang Yang, Yunxia Liu,
• Abstract summary: Face Super-Resolution aims to recover high-resolution (HR) face images from low-resolution (LR) ones. Existing approaches are not ideal due to their low reconstruction efficiency and insufficient utilization of prior information. This paper proposes a novel network architecture called W-Net to address this challenge.
• Score: 8.037821981254389
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Face Super-Resolution (FSR) aims to recover high-resolution (HR) face images from low-resolution (LR) ones. Despite the progress made by convolutional neural networks in FSR, the results of existing approaches are not ideal due to their low reconstruction efficiency and insufficient utilization of prior information. Considering that faces are highly structured objects, effectively leveraging facial priors to improve FSR results is a worthwhile endeavor. This paper proposes a novel network architecture called W-Net to address this challenge. W-Net leverages meticulously designed Parsing Block to fully exploit the resolution potential of LR image. We use this parsing map as an attention prior, effectively integrating information from both the parsing map and LR images. Simultaneously, we perform multiple fusions in various dimensions through the W-shaped network structure combined with the LPF(LR-Parsing Map Fusion Module). Additionally, we utilize a facial parsing graph as a mask, assigning different weights and loss functions to key facial areas to balance the performance of our reconstructed facial images between perceptual quality and pixel accuracy. We conducted extensive comparative experiments, not only limited to conventional facial super-resolution metrics but also extending to downstream tasks such as facial recognition and facial keypoint detection. The experiments demonstrate that W-Net exhibits outstanding performance in quantitative metrics, visual quality, and downstream tasks.

Related papers

• Improving Neural Surface Reconstruction with Feature Priors from Multi-View Image [87.00660347447494]
  Recent advancements in Neural Surface Reconstruction (NSR) have significantly improved multi-view reconstruction when coupled with volume rendering. We propose an investigation into feature-level consistent loss, aiming to harness valuable feature priors from diverse pretext visual tasks. Our results, analyzed on DTU and EPFL, reveal that feature priors from image matching and multi-view stereo datasets outperform other pretext tasks.
  arXiv  Detail & Related papers  (2024-08-04T16:09:46Z)
• Face Super-Resolution with Progressive Embedding of Multi-scale Face Priors [4.649637261351803]
  We propose a novel recurrent convolutional network based framework for face super-resolution. We take full advantage of the intermediate outputs of the recurrent network, and landmarks information and facial action units (AUs) information are extracted. Our proposed method significantly outperforms state-of-the-art FSR methods in terms of image quality and facial details restoration.
  arXiv  Detail & Related papers  (2022-10-12T08:16:52Z)
• Multi-Prior Learning via Neural Architecture Search for Blind Face Restoration [61.27907052910136]
  Blind Face Restoration (BFR) aims to recover high-quality face images from low-quality ones. Current methods still suffer from two major difficulties: 1) how to derive a powerful network architecture without extensive hand tuning; 2) how to capture complementary information from multiple facial priors in one network to improve restoration performance. We propose a Face Restoration Searching Network (FRSNet) to adaptively search the suitable feature extraction architecture within our specified search space.
  arXiv  Detail & Related papers  (2022-06-28T12:29:53Z)
• Hierarchical Similarity Learning for Aliasing Suppression Image Super-Resolution [64.15915577164894]
  A hierarchical image super-resolution network (HSRNet) is proposed to suppress the influence of aliasing. HSRNet achieves better quantitative and visual performance than other works, and remits the aliasing more effectively.
  arXiv  Detail & Related papers  (2022-06-07T14:55:32Z)
• CTCNet: A CNN-Transformer Cooperation Network for Face Image Super-Resolution [64.06360660979138]
  We propose an efficient CNN-Transformer Cooperation Network (CTCNet) for face super-resolution tasks. We first devise a novel Local-Global Feature Cooperation Module (LGCM), which is composed of a Facial Structure Attention Unit (FSAU) and a Transformer block. We then design an efficient Feature Refinement Module (FRM) to enhance the encoded features.
  arXiv  Detail & Related papers  (2022-04-19T06:38:29Z)
• TANet: A new Paradigm for Global Face Super-resolution via Transformer-CNN Aggregation Network [72.41798177302175]
  We propose a novel paradigm based on the self-attention mechanism (i.e., the core of Transformer) to fully explore the representation capacity of the facial structure feature. Specifically, we design a Transformer-CNN aggregation network (TANet) consisting of two paths, in which one path uses CNNs responsible for restoring fine-grained facial details. By aggregating the features from the above two paths, the consistency of global facial structure and fidelity of local facial detail restoration are strengthened simultaneously.
  arXiv  Detail & Related papers  (2021-09-16T18:15:07Z)
• Network Architecture Search for Face Enhancement [82.25775020564654]
  We present a multi-task face restoration network, called Network Architecture Search for Face Enhancement (NASFE) NASFE can enhance poor quality face images containing a single degradation (i.e. noise or blur) or multiple degradations (noise+blur+low-light)
  arXiv  Detail & Related papers  (2021-05-13T19:46:05Z)
• Learning Spatial Attention for Face Super-Resolution [28.60619685892613]
  General image super-resolution techniques have difficulties in recovering detailed face structures when applying to low resolution face images. Recent deep learning based methods tailored for face images have achieved improved performance by jointly trained with additional task such as face parsing and landmark prediction. We introduce a novel SPatial Attention Residual Network (SPARNet) built on our newly proposed Face Attention Units (FAUs) for face super-resolution.
  arXiv  Detail & Related papers  (2020-12-02T13:54:25Z)
• Joint Face Completion and Super-resolution using Multi-scale Feature Relation Learning [26.682678558621625]
  This paper proposes a multi-scale feature graph generative adversarial network (MFG-GAN) to implement the face restoration of images in which both degradation modes coexist. Based on the GAN, the MFG-GAN integrates the graph convolution and feature pyramid network to restore occluded low-resolution face images to non-occluded high-resolution face images. Experimental results on the public-domain CelebA and Helen databases show that the proposed approach outperforms state-of-the-art methods in performing face super-resolution (up to 4x or 8x) and face completion simultaneously.
  arXiv  Detail & Related papers  (2020-02-29T13:31:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.