Low-Rank Subspaces in GANs
- URL: http://arxiv.org/abs/2106.04488v1
- Date: Tue, 8 Jun 2021 16:16:32 GMT
- Title: Low-Rank Subspaces in GANs
- Authors: Jiapeng Zhu, Ruili Feng, Yujun Shen, Deli Zhao, Zhengjun Zha, Jingren
Zhou, Qifeng Chen
- Abstract summary: This work introduces low-rank subspaces that enable more precise control of GAN generation.
LowRankGAN is able to find the low-dimensional representation of attribute manifold.
Experiments on state-of-the-art GAN models (including StyleGAN2 and BigGAN) trained on various datasets demonstrate the effectiveness of our LowRankGAN.
- Score: 101.48350547067628
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: The latent space of a Generative Adversarial Network (GAN) has been shown to
encode rich semantics within some subspaces. To identify these subspaces,
researchers typically analyze the statistical information from a collection of
synthesized data, and the identified subspaces tend to control image attributes
globally (i.e., manipulating an attribute causes the change of an entire
image). By contrast, this work introduces low-rank subspaces that enable more
precise control of GAN generation. Concretely, given an arbitrary image and a
region of interest (e.g., eyes of face images), we manage to relate the latent
space to the image region with the Jacobian matrix and then use low-rank
factorization to discover steerable latent subspaces. There are three
distinguishable strengths of our approach that can be aptly called LowRankGAN.
First, compared to analytic algorithms in prior work, our low-rank
factorization of Jacobians is able to find the low-dimensional representation
of attribute manifold, making image editing more precise and controllable.
Second, low-rank factorization naturally yields a null space of attributes such
that moving the latent code within it only affects the outer region of
interest. Therefore, local image editing can be simply achieved by projecting
an attribute vector into the null space without relying on a spatial mask as
existing methods do. Third, our method can robustly work with a local region
from one image for analysis yet well generalize to other images, making it much
easy to use in practice. Extensive experiments on state-of-the-art GAN models
(including StyleGAN2 and BigGAN) trained on various datasets demonstrate the
effectiveness of our LowRankGAN.
Related papers
- Weakly-supervised deepfake localization in diffusion-generated images [4.548755617115687]
We propose a weakly-supervised localization problem based on the Xception network as the backbone architecture.
We show that the best performing detection method (based on local scores) is less sensitive to the looser supervision than to the mismatch in terms of dataset or generator.
arXiv Detail & Related papers (2023-11-08T10:27:36Z) - SC2GAN: Rethinking Entanglement by Self-correcting Correlated GAN Space [16.040942072859075]
Gene Networks that achieve following editing directions for one attribute could result in entangled changes with other attributes.
We propose a novel framework SC$2$GAN disentanglement by re-projecting low-density latent code samples in the original latent space.
arXiv Detail & Related papers (2023-10-10T14:42:32Z) - Spatial Latent Representations in Generative Adversarial Networks for
Image Generation [0.0]
We define a family of spatial latent spaces for StyleGAN2.
We show that our spaces are effective for image manipulation and encode semantic information well.
arXiv Detail & Related papers (2023-03-25T20:01:11Z) - Spatial Steerability of GANs via Self-Supervision from Discriminator [123.27117057804732]
We propose a self-supervised approach to improve the spatial steerability of GANs without searching for steerable directions in the latent space.
Specifically, we design randomly sampled Gaussian heatmaps to be encoded into the intermediate layers of generative models as spatial inductive bias.
During inference, users can interact with the spatial heatmaps in an intuitive manner, enabling them to edit the output image by adjusting the scene layout, moving, or removing objects.
arXiv Detail & Related papers (2023-01-20T07:36:29Z) - Towards Effective Image Manipulation Detection with Proposal Contrastive
Learning [61.5469708038966]
We propose Proposal Contrastive Learning (PCL) for effective image manipulation detection.
Our PCL consists of a two-stream architecture by extracting two types of global features from RGB and noise views respectively.
Our PCL can be easily adapted to unlabeled data in practice, which can reduce manual labeling costs and promote more generalizable features.
arXiv Detail & Related papers (2022-10-16T13:30:13Z) - Region-Based Semantic Factorization in GANs [67.90498535507106]
We present a highly efficient algorithm to factorize the latent semantics learned by Generative Adversarial Networks (GANs) concerning an arbitrary image region.
Through an appropriately defined generalized Rayleigh quotient, we solve such a problem without any annotations or training.
Experimental results on various state-of-the-art GAN models demonstrate the effectiveness of our approach.
arXiv Detail & Related papers (2022-02-19T17:46:02Z) - The Geometry of Deep Generative Image Models and its Applications [0.0]
Generative adversarial networks (GANs) have emerged as a powerful unsupervised method to model the statistical patterns of real-world data sets.
These networks are trained to map random inputs in their latent space to new samples representative of the learned data.
The structure of the latent space is hard to intuit due to its high dimensionality and the non-linearity of the generator.
arXiv Detail & Related papers (2021-01-15T07:57:33Z) - Positional Encoding as Spatial Inductive Bias in GANs [97.6622154941448]
SinGAN shows impressive capability in learning internal patch distribution despite its limited effective receptive field.
In this work, we show that such capability, to a large extent, is brought by the implicit positional encoding when using zero padding in the generators.
We propose a new multi-scale training strategy and demonstrate its effectiveness in the state-of-the-art unconditional generator StyleGAN2.
arXiv Detail & Related papers (2020-12-09T18:27:16Z) - Improving Inversion and Generation Diversity in StyleGAN using a
Gaussianized Latent Space [41.20193123974535]
Modern Generative Adversarial Networks are capable of creating artificial, photorealistic images from latent vectors living in a low-dimensional learned latent space.
We show that, under a simple nonlinear operation, the data distribution can be modeled as Gaussian and therefore expressed using sufficient statistics.
The resulting projections lie in smoother and better behaved regions of the latent space, as shown using performance for both real and generated images.
arXiv Detail & Related papers (2020-09-14T15:45:58Z) - Image Fine-grained Inpainting [89.17316318927621]
We present a one-stage model that utilizes dense combinations of dilated convolutions to obtain larger and more effective receptive fields.
To better train this efficient generator, except for frequently-used VGG feature matching loss, we design a novel self-guided regression loss.
We also employ a discriminator with local and global branches to ensure local-global contents consistency.
arXiv Detail & Related papers (2020-02-07T03:45:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.