GCFSR: a Generative and Controllable Face Super Resolution Method Without Facial and GAN Priors

• URL: http://arxiv.org/abs/2203.07319v1
• Date: Mon, 14 Mar 2022 17:22:19 GMT
• Title: GCFSR: a Generative and Controllable Face Super Resolution Method Without Facial and GAN Priors
• Authors: Jingwen He, Wu Shi, Kai Chen, Lean Fu, Chao Dong
• Abstract summary: GCFSR is a generative and controllable face SR framework. It reconstructs images with faithful identity information without any additional priors. It can outperform state-of-the-art methods for large upscaling factors.
• Score: 23.74675310360883
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Face image super resolution (face hallucination) usually relies on facial priors to restore realistic details and preserve identity information. Recent advances can achieve impressive results with the help of GAN prior. They either design complicated modules to modify the fixed GAN prior or adopt complex training strategies to finetune the generator. In this work, we propose a generative and controllable face SR framework, called GCFSR, which can reconstruct images with faithful identity information without any additional priors. Generally, GCFSR has an encoder-generator architecture. Two modules called style modulation and feature modulation are designed for the multi-factor SR task. The style modulation aims to generate realistic face details and the feature modulation dynamically fuses the multi-level encoded features and the generated ones conditioned on the upscaling factor. The simple and elegant architecture can be trained from scratch in an end-to-end manner. For small upscaling factors (<=8), GCFSR can produce surprisingly good results with only adversarial loss. After adding L1 and perceptual losses, GCFSR can outperform state-of-the-art methods for large upscaling factors (16, 32, 64). During the test phase, we can modulate the generative strength via feature modulation by changing the conditional upscaling factor continuously to achieve various generative effects.

Related papers

• Specularity Factorization for Low-Light Enhancement [2.7961648901433134]
  We present a new additive image factorization technique that treats images to be composed of multiple latent components. Our model-driven em RSFNet estimates these factors by unrolling the optimization into network layers. The resultant factors are interpretable by design and can be fused for different image enhancement tasks via a network or combined directly by the user.
  arXiv  Detail & Related papers  (2024-04-02T14:41:42Z)
• Boosting Flow-based Generative Super-Resolution Models via Learned Prior [8.557017814978334]
  Flow-based super-resolution (SR) models have demonstrated astonishing capabilities in generating high-quality images. These methods encounter several challenges during image generation, such as grid artifacts, exploding inverses, and suboptimal results due to a fixed sampling temperature. This work introduces a conditional learned prior to the inference phase of a flow-based SR model.
  arXiv  Detail & Related papers  (2024-03-16T18:04:12Z)
• BFRFormer: Transformer-based generator for Real-World Blind Face Restoration [37.77996097891398]
  We propose a Transformer-based blind face restoration method, named BFRFormer, to reconstruct images with more identity-preserved details in an end-to-end manner. Our method outperforms state-of-the-art methods on a synthetic dataset and four real-world datasets.
  arXiv  Detail & Related papers  (2024-02-29T02:31:54Z)
• Learning Detail-Structure Alternative Optimization for Blind Super-Resolution [69.11604249813304]
  We propose an effective and kernel-free network, namely DSSR, which enables recurrent detail-structure alternative optimization without blur kernel prior incorporation for blind SR. In our DSSR, a detail-structure modulation module (DSMM) is built to exploit the interaction and collaboration of image details and structures. Our method achieves the state-of-the-art against existing methods.
  arXiv  Detail & Related papers  (2022-12-03T14:44:17Z)
• RRSR:Reciprocal Reference-based Image Super-Resolution with Progressive Feature Alignment and Selection [66.08293086254851]
  We propose a reciprocal learning framework to reinforce the learning of a RefSR network. The newly proposed module aligns reference-input images at multi-scale feature spaces and performs reference-aware feature selection. We empirically show that multiple recent state-of-the-art RefSR models can be consistently improved with our reciprocal learning paradigm.
  arXiv  Detail & Related papers  (2022-11-08T12:39:35Z)
• StyleSwap: Style-Based Generator Empowers Robust Face Swapping [90.05775519962303]
  We introduce a concise and effective framework named StyleSwap. Our core idea is to leverage a style-based generator to empower high-fidelity and robust face swapping. We identify that with only minimal modifications, a StyleGAN2 architecture can successfully handle the desired information from both source and target.
  arXiv  Detail & Related papers  (2022-09-27T16:35:16Z)
• Latent Multi-Relation Reasoning for GAN-Prior based Image Super-Resolution [61.65012981435095]
  LAREN is a graph-based disentanglement that constructs a superior disentangled latent space via hierarchical multi-relation reasoning. We show that LAREN achieves superior large-factor image SR and outperforms the state-of-the-art consistently across multiple benchmarks.
  arXiv  Detail & Related papers  (2022-08-04T19:45:21Z)
• CTCNet: A CNN-Transformer Cooperation Network for Face Image Super-Resolution [64.06360660979138]
  We propose an efficient CNN-Transformer Cooperation Network (CTCNet) for face super-resolution tasks. We first devise a novel Local-Global Feature Cooperation Module (LGCM), which is composed of a Facial Structure Attention Unit (FSAU) and a Transformer block. We then design an efficient Feature Refinement Module (FRM) to enhance the encoded features.
  arXiv  Detail & Related papers  (2022-04-19T06:38:29Z)
• Decision Transformer: Reinforcement Learning via Sequence Modeling [102.86873656751489]
  We present a framework that abstracts Reinforcement Learning (RL) as a sequence modeling problem. We present Decision Transformer, an architecture that casts the problem of RL as conditional sequence modeling. Despite its simplicity, Decision Transformer matches or exceeds the performance of state-of-the-art offline RL baselines on Atari, OpenAI Gym, and Key-to-Door tasks.
  arXiv  Detail & Related papers  (2021-06-02T17:53:39Z)
• Toward Interactive Modulation for Photo-Realistic Image Restoration [16.610981587637102]
  Modulating image restoration level aims to generate a restored image by altering a factor that represents the restoration strength. This paper presents a Controllable Unet Generative Adversarial Network (CUGAN) to generate high-frequency textures in the modulation tasks.
  arXiv  Detail & Related papers  (2021-05-07T07:05:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.