Force-in-domain GAN inversion

• URL: http://arxiv.org/abs/2107.06050v2
• Date: Wed, 14 Jul 2021 01:42:15 GMT
• Title: Force-in-domain GAN inversion
• Authors: Guangjie Leng, Yekun Zhu and Zhi-Qin John Xu
• Abstract summary: Various semantics emerge in the latent space of Generative Adversarial Networks (GANs) when being trained to generate images. An in-domain GAN inversion approach is recently proposed to constraint the inverted code within the latent space. We propose a force-in-domain GAN based on the in-domain GAN, which utilizes a discriminator to force the inverted code within the latent space.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Empirical works suggest that various semantics emerge in the latent space of Generative Adversarial Networks (GANs) when being trained to generate images. To perform real image editing, it requires an accurate mapping from the real image to the latent space to leveraging these learned semantics, which is important yet difficult. An in-domain GAN inversion approach is recently proposed to constraint the inverted code within the latent space by forcing the reconstructed image obtained from the inverted code within the real image space. Empirically, we find that the inverted code by the in-domain GAN can deviate from the latent space significantly. To solve this problem, we propose a force-in-domain GAN based on the in-domain GAN, which utilizes a discriminator to force the inverted code within the latent space. The force-in-domain GAN can also be interpreted by a cycle-GAN with slight modification. Extensive experiments show that our force-in-domain GAN not only reconstructs the target image at the pixel level, but also align the inverted code with the latent space well for semantic editing.

Related papers

• In-Domain GAN Inversion for Faithful Reconstruction and Editability [132.68255553099834]
  We propose in-domain GAN inversion, which consists of a domain-guided domain-regularized and a encoder to regularize the inverted code in the native latent space of the pre-trained GAN model. We make comprehensive analyses on the effects of the encoder structure, the starting inversion point, as well as the inversion parameter space, and observe the trade-off between the reconstruction quality and the editing property.
  arXiv  Detail & Related papers  (2023-09-25T08:42:06Z)
• Editing Out-of-domain GAN Inversion via Differential Activations [56.62964029959131]
  We propose a novel GAN prior based editing framework to tackle the out-of-domain inversion problem with a composition-decomposition paradigm. With the aid of the generated Diff-CAM mask, a coarse reconstruction can intuitively be composited by the paired original and edited images. In the decomposition phase, we further present a GAN prior based deghosting network for separating the final fine edited image from the coarse reconstruction.
  arXiv  Detail & Related papers  (2022-07-17T10:34:58Z)
• GAN Inversion for Out-of-Range Images with Geometric Transformations [22.914126221037222]
  We propose BDInvert, a novel GAN inversion approach to semantic editing of out-of-range images. Our experiments show that BDInvert effectively supports semantic editing of out-of-range images with geometric transformations.
  arXiv  Detail & Related papers  (2021-08-20T04:38:40Z)
• GAN Inversion: A Survey [125.62848237531945]
  GAN inversion aims to invert a given image back into the latent space of a pretrained GAN model. GAN inversion plays an essential role in enabling the pretrained GAN models such as StyleGAN and BigGAN to be used for real image editing applications.
  arXiv  Detail & Related papers  (2021-01-14T14:11:00Z)
• Positional Encoding as Spatial Inductive Bias in GANs [97.6622154941448]
  SinGAN shows impressive capability in learning internal patch distribution despite its limited effective receptive field. In this work, we show that such capability, to a large extent, is brought by the implicit positional encoding when using zero padding in the generators. We propose a new multi-scale training strategy and demonstrate its effectiveness in the state-of-the-art unconditional generator StyleGAN2.
  arXiv  Detail & Related papers  (2020-12-09T18:27:16Z)
• Image-to-image Mapping with Many Domains by Sparse Attribute Transfer [71.28847881318013]
  Unsupervised image-to-image translation consists of learning a pair of mappings between two domains without known pairwise correspondences between points. Current convention is to approach this task with cycle-consistent GANs. We propose an alternate approach that directly restricts the generator to performing a simple sparse transformation in a latent layer.
  arXiv  Detail & Related papers  (2020-06-23T19:52:23Z)
• In-Domain GAN Inversion for Real Image Editing [56.924323432048304]
  A common practice of feeding a real image to a trained GAN generator is to invert it back to a latent code. Existing inversion methods typically focus on reconstructing the target image by pixel values yet fail to land the inverted code in the semantic domain of the original latent space. We propose an in-domain GAN inversion approach, which faithfully reconstructs the input image and ensures the inverted code to be semantically meaningful for editing.
  arXiv  Detail & Related papers  (2020-03-31T18:20:18Z)
• Optimizing Generative Adversarial Networks for Image Super Resolution via Latent Space Regularization [4.529132742139768]
  Generative Adversarial Networks (GANs) try to learn the distribution of the real images in the manifold to generate samples that look real. We probe for ways to alleviate these problems for supervised GANs in this paper.
  arXiv  Detail & Related papers  (2020-01-22T16:27:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.