Face sketch to photo translation using generative adversarial networks
- URL: http://arxiv.org/abs/2110.12290v1
- Date: Sat, 23 Oct 2021 20:01:20 GMT
- Title: Face sketch to photo translation using generative adversarial networks
- Authors: Nastaran Moradzadeh Farid, Maryam Saeedi Fard, Ahmad Nickabadi
- Abstract summary: We use a pre-trained face photo generating model to synthesize high-quality natural face photos.
We train a network to map the facial features extracted from the input sketch to a vector in the latent space of the face generating model.
The proposed model achieved 0.655 in the SSIM index and 97.59% rank-1 face recognition rate.
- Score: 1.0312968200748118
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Translating face sketches to photo-realistic faces is an interesting and
essential task in many applications like law enforcement and the digital
entertainment industry. One of the most important challenges of this task is
the inherent differences between the sketch and the real image such as the lack
of color and details of the skin tissue in the sketch. With the advent of
adversarial generative models, an increasing number of methods have been
proposed for sketch-to-image synthesis. However, these models still suffer from
limitations such as the large number of paired data required for training, the
low resolution of the produced images, or the unrealistic appearance of the
generated images. In this paper, we propose a method for converting an input
facial sketch to a colorful photo without the need for any paired dataset. To
do so, we use a pre-trained face photo generating model to synthesize
high-quality natural face photos and employ an optimization procedure to keep
high-fidelity to the input sketch. We train a network to map the facial
features extracted from the input sketch to a vector in the latent space of the
face generating model. Also, we study different optimization criteria and
compare the results of the proposed model with those of the state-of-the-art
models quantitatively and qualitatively. The proposed model achieved 0.655 in
the SSIM index and 97.59% rank-1 face recognition rate with higher quality of
the produced images.
Related papers
- OSDFace: One-Step Diffusion Model for Face Restoration [72.5045389847792]
Diffusion models have demonstrated impressive performance in face restoration.
We propose OSDFace, a novel one-step diffusion model for face restoration.
Results demonstrate that OSDFace surpasses current state-of-the-art (SOTA) methods in both visual quality and quantitative metrics.
arXiv Detail & Related papers (2024-11-26T07:07:48Z) - Multi-Style Facial Sketch Synthesis through Masked Generative Modeling [17.313050611750413]
We propose a lightweight end-to-end synthesis model that efficiently converts images to corresponding multi-stylized sketches.
In this study, we overcome the issue of data insufficiency by incorporating semi-supervised learning into the training process.
Our method consistently outperforms previous algorithms across multiple benchmarks.
arXiv Detail & Related papers (2024-08-22T13:45:04Z) - Improving face generation quality and prompt following with synthetic captions [57.47448046728439]
We introduce a training-free pipeline designed to generate accurate appearance descriptions from images of people.
We then use these synthetic captions to fine-tune a text-to-image diffusion model.
Our results demonstrate that this approach significantly improves the model's ability to generate high-quality, realistic human faces.
arXiv Detail & Related papers (2024-05-17T15:50:53Z) - FaceChain: A Playground for Human-centric Artificial Intelligence
Generated Content [36.48960592782015]
FaceChain is a personalized portrait generation framework that combines a series of customized image-generation model and a rich set of face-related perceptual understanding models.
We inject several SOTA face models into the generation procedure, achieving a more efficient label-tagging, data-processing, and model post-processing compared to previous solutions.
Based on FaceChain, we further develop several applications to build a broader playground for better showing its value, including virtual try on and 2D talking head.
arXiv Detail & Related papers (2023-08-28T02:20:44Z) - DreamIdentity: Improved Editability for Efficient Face-identity
Preserved Image Generation [69.16517915592063]
We propose a novel face-identity encoder to learn an accurate representation of human faces.
We also propose self-augmented editability learning to enhance the editability of models.
Our methods can generate identity-preserved images under different scenes at a much faster speed.
arXiv Detail & Related papers (2023-07-01T11:01:17Z) - DiffSketching: Sketch Control Image Synthesis with Diffusion Models [10.172753521953386]
Deep learning models for sketch-to-image synthesis need to overcome the distorted input sketch without visual details.
Our model matches sketches through the cross domain constraints, and uses a classifier to guide the image synthesis more accurately.
Our model can beat GAN-based method in terms of generation quality and human evaluation, and does not rely on massive sketch-image datasets.
arXiv Detail & Related papers (2023-05-30T07:59:23Z) - Person Image Synthesis via Denoising Diffusion Model [116.34633988927429]
We show how denoising diffusion models can be applied for high-fidelity person image synthesis.
Our results on two large-scale benchmarks and a user study demonstrate the photorealism of our proposed approach under challenging scenarios.
arXiv Detail & Related papers (2022-11-22T18:59:50Z) - Quality Metric Guided Portrait Line Drawing Generation from Unpaired
Training Data [88.78171717494688]
We propose a novel method to automatically transform face photos to portrait drawings using unpaired training data.
Our method can (1) learn to generate high quality portrait drawings in multiple styles using a single network and (2) generate portrait drawings in a "new style" unseen in the training data.
arXiv Detail & Related papers (2022-02-08T06:49:57Z) - DeepFacePencil: Creating Face Images from Freehand Sketches [77.00929179469559]
Existing image-to-image translation methods require a large-scale dataset of paired sketches and images for supervision.
We propose DeepFacePencil, an effective tool that is able to generate photo-realistic face images from hand-drawn sketches.
arXiv Detail & Related papers (2020-08-31T03:35:21Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.