GAN Inversion: A Survey

• URL: http://arxiv.org/abs/2101.05278v2
• Date: Mon, 8 Mar 2021 07:05:20 GMT
• Title: GAN Inversion: A Survey
• Authors: Weihao Xia, Yulun Zhang, Yujiu Yang, Jing-Hao Xue, Bolei Zhou, Ming-Hsuan Yang
• Abstract summary: GAN inversion aims to invert a given image back into the latent space of a pretrained GAN model. GAN inversion plays an essential role in enabling the pretrained GAN models such as StyleGAN and BigGAN to be used for real image editing applications.
• Score: 125.62848237531945
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: GAN inversion aims to invert a given image back into the latent space of a pretrained GAN model, for the image to be faithfully reconstructed from the inverted code by the generator. As an emerging technique to bridge the real and fake image domains, GAN inversion plays an essential role in enabling the pretrained GAN models such as StyleGAN and BigGAN to be used for real image editing applications. Meanwhile, GAN inversion also provides insights on the interpretation of GAN's latent space and how the realistic images can be generated. In this paper, we provide an overview of GAN inversion with a focus on its recent algorithms and applications. We cover important techniques of GAN inversion and their applications to image restoration and image manipulation. We further elaborate on some trends and challenges for future directions.

Related papers

• EGAIN: Extended GAn INversion [5.602947425285195]
  Generative Adversarial Networks (GANs) have witnessed significant advances in recent years. Recent GANs have proven to encode features in a disentangled latent space. GAN inversion opens the door for the manipulation of facial semantics of real face images.
  arXiv  Detail & Related papers  (2023-12-22T23:25:17Z)
• In-Domain GAN Inversion for Faithful Reconstruction and Editability [132.68255553099834]
  We propose in-domain GAN inversion, which consists of a domain-guided domain-regularized and a encoder to regularize the inverted code in the native latent space of the pre-trained GAN model. We make comprehensive analyses on the effects of the encoder structure, the starting inversion point, as well as the inversion parameter space, and observe the trade-off between the reconstruction quality and the editing property.
  arXiv  Detail & Related papers  (2023-09-25T08:42:06Z)
• JoIN: Joint GANs Inversion for Intrinsic Image Decomposition [16.02463667910604]
  We propose to solve ill-posed inverse imaging problems using a bank of Generative Adversarial Networks (GAN) Our method builds on the demonstrated success of GANs to capture complex image distributions.
  arXiv  Detail & Related papers  (2023-05-18T22:09:32Z)
• UnGANable: Defending Against GAN-based Face Manipulation [69.90981797810348]
  Deepfakes pose severe threats of visual misinformation to our society. One representative deepfake application is face manipulation that modifies a victim's facial attributes in an image. We propose the first defense system, namely UnGANable, against GAN-inversion-based face manipulation.
  arXiv  Detail & Related papers  (2022-10-03T14:20:01Z)
• StyleGAN-induced data-driven regularization for inverse problems [2.5138572116292686]
  Recent advances in generative adversarial networks (GANs) have opened up the possibility of generating high-resolution images that were impossible to produce previously. We develop a framework that utilizes the full potential of a pre-trained StyleGAN2 generator for constructing the prior distribution on the underlying image. Considering the inverse problems of image inpainting and super-resolution, we demonstrate that the proposed approach is competitive with, and sometimes superior to, state-of-the-art GAN-based image reconstruction methods.
  arXiv  Detail & Related papers  (2021-10-07T22:25:30Z)
• Cycle-Consistent Inverse GAN for Text-to-Image Synthesis [101.97397967958722]
  We propose a novel unified framework of Cycle-consistent Inverse GAN for both text-to-image generation and text-guided image manipulation tasks. We learn a GAN inversion model to convert the images back to the GAN latent space and obtain the inverted latent codes for each image. In the text-guided optimization module, we generate images with the desired semantic attributes by optimizing the inverted latent codes.
  arXiv  Detail & Related papers  (2021-08-03T08:38:16Z)
• In-Domain GAN Inversion for Real Image Editing [56.924323432048304]
  A common practice of feeding a real image to a trained GAN generator is to invert it back to a latent code. Existing inversion methods typically focus on reconstructing the target image by pixel values yet fail to land the inverted code in the semantic domain of the original latent space. We propose an in-domain GAN inversion approach, which faithfully reconstructs the input image and ensures the inverted code to be semantically meaningful for editing.
  arXiv  Detail & Related papers  (2020-03-31T18:20:18Z)
• Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation [181.08127307338654]
  This work presents an effective way to exploit the image prior captured by a generative adversarial network (GAN) trained on large-scale natural images. The deep generative prior (DGP) provides compelling results to restore missing semantics, e.g., color, patch, resolution, of various degraded images.
  arXiv  Detail & Related papers  (2020-03-30T17:45:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.