Retrieve in Style: Unsupervised Facial Feature Transfer and Retrieval
- URL: http://arxiv.org/abs/2107.06256v1
- Date: Tue, 13 Jul 2021 17:31:34 GMT
- Title: Retrieve in Style: Unsupervised Facial Feature Transfer and Retrieval
- Authors: Min Jin Chong, Wen-Sheng Chu, Abhishek Kumar
- Abstract summary: Retrieve in Style (RIS) is an unsupervised framework for fine-grained facial feature transfer and retrieval on real images.
RIS achieves both high-fidelity feature transfers and accurate fine-grained retrievals on real images.
- Score: 17.833454714281757
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We present Retrieve in Style (RIS), an unsupervised framework for
fine-grained facial feature transfer and retrieval on real images. Recent work
shows that it is possible to learn a catalog that allows local semantic
transfers of facial features on generated images by capitalizing on the
disentanglement property of the StyleGAN latent space. RIS improves existing
art on: 1) feature disentanglement and allows for challenging transfers (i.e.,
hair and pose) that were not shown possible in SoTA methods. 2) eliminating the
need for per-image hyperparameter tuning, and for computing a catalog over a
large batch of images. 3) enabling face retrieval using the proposed facial
features (e.g., eyes), and to our best knowledge, is the first work to retrieve
face images at the fine-grained level. 4) robustness and natural application to
real images. Our qualitative and quantitative analyses show RIS achieves both
high-fidelity feature transfers and accurate fine-grained retrievals on real
images. We discuss the responsible application of RIS.
Related papers
- One-shot Neural Face Reenactment via Finding Directions in GAN's Latent
Space [37.357842761713705]
We present a framework for neural face/head reenactment whose goal is to transfer the 3D head orientation and expression of a target face to a source face.
Our method features several favorable properties including using a single source image (one-shot) and enabling cross-person reenactment.
arXiv Detail & Related papers (2024-02-05T22:12:42Z) - HyperReenact: One-Shot Reenactment via Jointly Learning to Refine and
Retarget Faces [47.27033282706179]
We present our method for neural face reenactment, called HyperReenact, that aims to generate realistic talking head images of a source identity.
Our method operates under the one-shot setting (i.e., using a single source frame) and allows for cross-subject reenactment, without requiring subject-specific fine-tuning.
We compare our method both quantitatively and qualitatively against several state-of-the-art techniques on the standard benchmarks of VoxCeleb1 and VoxCeleb2.
arXiv Detail & Related papers (2023-07-20T11:59:42Z) - Finding Directions in GAN's Latent Space for Neural Face Reenactment [45.67273942952348]
This paper is on face/head reenactment where the goal is to transfer the facial pose (3D head orientation and expression) of a target face to a source face.
We take a different approach, bypassing the training of such networks, by using (fine-tuned) pre-trained GANs.
We show that by embedding real images in the GAN latent space, our method can be successfully used for the reenactment of real-world faces.
arXiv Detail & Related papers (2022-01-31T19:14:03Z) - Pro-UIGAN: Progressive Face Hallucination from Occluded Thumbnails [53.080403912727604]
We propose a multi-stage Progressive Upsampling and Inpainting Generative Adversarial Network, dubbed Pro-UIGAN.
It exploits facial geometry priors to replenish and upsample (8*) the occluded and tiny faces.
Pro-UIGAN achieves visually pleasing HR faces, reaching superior performance in downstream tasks.
arXiv Detail & Related papers (2021-08-02T02:29:24Z) - Network Architecture Search for Face Enhancement [82.25775020564654]
We present a multi-task face restoration network, called Network Architecture Search for Face Enhancement (NASFE)
NASFE can enhance poor quality face images containing a single degradation (i.e. noise or blur) or multiple degradations (noise+blur+low-light)
arXiv Detail & Related papers (2021-05-13T19:46:05Z) - Foreground-guided Facial Inpainting with Fidelity Preservation [7.5089719291325325]
We propose a foreground-guided facial inpainting framework that can extract and generate facial features using convolutional neural network layers.
Specifically, we propose a new loss function with semantic capability reasoning of facial expressions, natural and unnatural features (make-up)
Our proposed method achieved comparable quantitative results when compare to the state of the art but qualitatively, it demonstrated high-fidelity preservation of facial components.
arXiv Detail & Related papers (2021-05-07T15:50:58Z) - S2FGAN: Semantically Aware Interactive Sketch-to-Face Translation [11.724779328025589]
This paper proposes a sketch-to-image generation framework called S2FGAN.
We employ two latent spaces to control the face appearance and adjust the desired attributes of the generated face.
Our method successfully outperforms state-of-the-art methods on attribute manipulation by exploiting greater control of attribute intensity.
arXiv Detail & Related papers (2020-11-30T13:42:39Z) - Bridging Composite and Real: Towards End-to-end Deep Image Matting [88.79857806542006]
We study the roles of semantics and details for image matting.
We propose a novel Glance and Focus Matting network (GFM), which employs a shared encoder and two separate decoders.
Comprehensive empirical studies have demonstrated that GFM outperforms state-of-the-art methods.
arXiv Detail & Related papers (2020-10-30T10:57:13Z) - From A Glance to "Gotcha": Interactive Facial Image Retrieval with
Progressive Relevance Feedback [72.29919762941029]
We propose an end-to-end framework to retrieve facial images with relevance feedback progressively provided by the witness.
With no need of any extra annotations, our model can be applied at the cost of a little response effort.
arXiv Detail & Related papers (2020-07-30T18:46:25Z) - HiFaceGAN: Face Renovation via Collaborative Suppression and
Replenishment [63.333407973913374]
"Face Renovation"(FR) is a semantic-guided generation problem.
"HiFaceGAN" is a multi-stage framework containing several nested CSR units.
experiments on both synthetic and real face images have verified the superior performance of HiFaceGAN.
arXiv Detail & Related papers (2020-05-11T11:33:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.