Multiple Exemplars-based Hallucinationfor Face Super-resolution and Editing

• URL: http://arxiv.org/abs/2009.07827v3
• Date: Thu, 14 Jan 2021 09:55:02 GMT
• Title: Multiple Exemplars-based Hallucinationfor Face Super-resolution and Editing
• Authors: Kaili Wang, Jose Oramas, Tinne Tuytelaars
• Abstract summary: Given a really low-resolution input image of a face, the goal of this paper is to reconstruct a high-resolution version thereof. We explore the use of a set of exemplars, i.e. other high-resolution images of the same person. To combine the information from multiple exemplars effectively, we introduce a pixel-wise weight generation module.
• Score: 38.257982713474874
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Given a really low-resolution input image of a face (say 16x16 or 8x8 pixels), the goal of this paper is to reconstruct a high-resolution version thereof. This, by itself, is an ill-posed problem, as the high-frequency information is missing in the low-resolution input and needs to be hallucinated, based on prior knowledge about the image content. Rather than relying on a generic face prior, in this paper, we explore the use of a set of exemplars, i.e. other high-resolution images of the same person. These guide the neural network as we condition the output on them. Multiple exemplars work better than a single one. To combine the information from multiple exemplars effectively, we introduce a pixel-wise weight generation module. Besides standard face super-resolution, our method allows to perform subtle face editing simply by replacing the exemplars with another set with different facial features. A user study is conducted and shows the super-resolved images can hardly be distinguished from real images on the CelebA dataset. A qualitative comparison indicates our model outperforms methods proposed in the literature on the CelebA and WebFace dataset.

Related papers

• Multi-Feature Aggregation in Diffusion Models for Enhanced Face Super-Resolution [6.055006354743854]
  We develop an algorithm that utilize a low-resolution image combined with features extracted from multiple low-quality images to generate a super-resolved image. Unlike other algorithms, our approach recovers facial features without explicitly providing attribute information. This is the first time multi-features combined with low-resolution images are used as conditioners to generate more reliable super-resolution images.
  arXiv  Detail & Related papers  (2024-08-27T20:08:33Z)
• Arc2Face: A Foundation Model for ID-Consistent Human Faces [95.00331107591859]
  Arc2Face is an identity-conditioned face foundation model. It can generate diverse photo-realistic images with an unparalleled degree of face similarity than existing models.
  arXiv  Detail & Related papers  (2024-03-18T10:32:51Z)
• Face0: Instantaneously Conditioning a Text-to-Image Model on a Face [3.5150821092068383]
  We present Face0, a novel way to instantaneously condition a text-to-image generation model on a face. We augment a dataset of annotated images with embeddings of the included faces and train an image generation model, on the augmented dataset. Our method achieves pleasing results, is remarkably simple, extremely fast, and equips the underlying model with new capabilities.
  arXiv  Detail & Related papers  (2023-06-11T09:52:03Z)
• HIME: Efficient Headshot Image Super-Resolution with Multiple Exemplars [11.81364643562714]
  We propose an efficient Headshot Image Super-Resolution with Multiple Exemplars network (HIME) method. Compared with previous methods, our network can effectively handle the misalignment between the input and the reference. We also propose a correlation loss that provides a rich representation of the local texture in a controllable spatial range.
  arXiv  Detail & Related papers  (2022-03-28T16:13:28Z)
• Face sketch to photo translation using generative adversarial networks [1.0312968200748118]
  We use a pre-trained face photo generating model to synthesize high-quality natural face photos. We train a network to map the facial features extracted from the input sketch to a vector in the latent space of the face generating model. The proposed model achieved 0.655 in the SSIM index and 97.59% rank-1 face recognition rate.
  arXiv  Detail & Related papers  (2021-10-23T20:01:20Z)
• LR-to-HR Face Hallucination with an Adversarial Progressive Attribute-Induced Network [67.64536397027229]
  Face super-resolution is a challenging and highly ill-posed problem. We propose an end-to-end progressive learning framework incorporating facial attributes. We show that the proposed approach can yield satisfactory face hallucination images outperforming other state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-09-29T19:50:45Z)
• Deep Learning-based Face Super-resolution: A Survey [78.11274281686246]
  Face super-resolution, also known as face hallucination, is a domain-specific image super-resolution problem. To date, few summaries of the studies on the deep learning-based face super-resolution are available. In this survey, we present a comprehensive review of deep learning techniques in face super-resolution in a systematic manner.
  arXiv  Detail & Related papers  (2021-01-11T08:17:11Z)
• Unsupervised Real Image Super-Resolution via Generative Variational AutoEncoder [47.53609520395504]
  We revisit the classic example based image super-resolution approaches and come up with a novel generative model for perceptual image super-resolution. We propose a joint image denoising and super-resolution model via Variational AutoEncoder. With the aid of the discriminator, an additional overhead of super-resolution subnetwork is attached to super-resolve the denoised image with photo-realistic visual quality.
  arXiv  Detail & Related papers  (2020-04-27T13:49:36Z)
• PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models [77.32079593577821]
  PULSE (Photo Upsampling via Latent Space Exploration) generates high-resolution, realistic images at resolutions previously unseen in the literature. Our method outperforms state-of-the-art methods in perceptual quality at higher resolutions and scale factors than previously possible.
  arXiv  Detail & Related papers  (2020-03-08T16:44:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.