Do Not Escape From the Manifold: Discovering the Local Coordinates on the Latent Space of GANs

• URL: http://arxiv.org/abs/2106.06959v1
• Date: Sun, 13 Jun 2021 10:29:42 GMT
• Title: Do Not Escape From the Manifold: Discovering the Local Coordinates on the Latent Space of GANs
• Authors: Jaewoong Choi, Changyeon Yoon, Junho Lee, Jung Ho Park, Geonho Hwang, Myungjoo Kang
• Abstract summary: We propose a method to find local-geometry-aware traversal directions on the intermediate latent space of Generative Adversarial Networks (GANs) Motivated by the intrinsic sparsity of the latent space, the basis is discovered by solving the low-rank approximation problem of the differential of the partial network.
• Score: 7.443321740418409
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we propose a method to find local-geometry-aware traversal directions on the intermediate latent space of Generative Adversarial Networks (GANs). These directions are defined as an ordered basis of tangent space at a latent code. Motivated by the intrinsic sparsity of the latent space, the basis is discovered by solving the low-rank approximation problem of the differential of the partial network. Moreover, the local traversal basis leads to a natural iterative traversal on the latent space. Iterative Curve-Traversal shows stable traversal on images, since the trajectory of latent code stays close to the latent space even under the strong perturbations compared to the linear traversal. This stability provides far more diverse variations of the given image. Although the proposed method can be applied to various GAN models, we focus on the W-space of the StyleGAN2, which is renowned for showing the better disentanglement of the latent factors of variation. Our quantitative and qualitative analysis provides evidence showing that the W-space is still globally warped while showing a certain degree of global consistency of interpretable variation. In particular, we introduce some metrics on the Grassmannian manifolds to quantify the global warpage of the W-space and the subspace traversal to test the stability of traversal directions.

Related papers

• Unsupervised Panoptic Interpretation of Latent Spaces in GANs Using Space-Filling Vector Quantization [9.181917968017258]
  Generative adversarial networks (GANs) learn a latent space whose samples can be mapped to real-world images. Some earlier supervised methods aim to create an interpretable latent space or discover interpretable directions. We propose using a modification of vector quantization called space-filling vector quantization (SFVQ), which quantizes the data on a piece-wise linear curve.
  arXiv  Detail & Related papers  (2024-10-27T19:56:02Z)
• SGFormer: Spherical Geometry Transformer for 360 Depth Estimation [54.13459226728249]
  Panoramic distortion poses a significant challenge in 360 depth estimation. We propose a spherical geometry transformer, named SGFormer, to address the above issues. We also present a query-based global conditional position embedding to compensate for spatial structure at varying resolutions.
  arXiv  Detail & Related papers  (2024-04-23T12:36:24Z)
• Locality-preserving Directions for Interpreting the Latent Space of Satellite Image GANs [20.010911311234718]
  We present a locality-aware method for interpreting the latent space of wavelet-based Generative Adversarial Networks (GANs) By focusing on preserving locality, the proposed method is able to decompose the weight-space of pre-trained GANs and recover interpretable directions.
  arXiv  Detail & Related papers  (2023-09-26T12:29:36Z)
• Discovering Class-Specific GAN Controls for Semantic Image Synthesis [73.91655061467988]
  We propose a novel method for finding spatially disentangled class-specific directions in the latent space of pretrained SIS models. We show that the latent directions found by our method can effectively control the local appearance of semantic classes.
  arXiv  Detail & Related papers  (2022-12-02T21:39:26Z)
• Analyzing the Latent Space of GAN through Local Dimension Estimation [4.688163910878411]
  style-based GANs (StyleGANs) in high-fidelity image synthesis have motivated research to understand the semantic properties of their latent spaces. We propose a local dimension estimation algorithm for arbitrary intermediate layers in a pre-trained GAN model. Our proposed metric, called Distortion, measures an inconsistency of intrinsic space on the learned latent space.
  arXiv  Detail & Related papers  (2022-05-26T06:36:06Z)
• Dencentralized learning in the presence of low-rank noise [57.18977364494388]
  Observations collected by agents in a network may be unreliable due to observation noise or interference. This paper proposes a distributed algorithm that allows each node to improve the reliability of its own observation.
  arXiv  Detail & Related papers  (2022-03-18T09:13:57Z)
• Few Shot Generative Model Adaption via Relaxed Spatial Structural Alignment [130.84010267004803]
  Training a generative adversarial network (GAN) with limited data has been a challenging task. A feasible solution is to start with a GAN well-trained on a large scale source domain and adapt it to the target domain with a few samples, termed as few shot generative model adaption. We propose a relaxed spatial structural alignment method to calibrate the target generative models during the adaption.
  arXiv  Detail & Related papers  (2022-03-06T14:26:25Z)
• WarpedGANSpace: Finding non-linear RBF paths in GAN latent space [44.7091944340362]
  This work addresses the problem of discovering, in an unsupervised manner, interpretable paths in the latent space of pretrained GANs. We learn non-linear warpings on the latent space, each one parametrized by a set of RBF-based latent space warping functions. We show that linear paths can be derived as a special case of our method, and show experimentally that non-linear paths in the latent space lead to steeper, more disentangled and interpretable changes in the image space.
  arXiv  Detail & Related papers  (2021-09-27T21:29:35Z)
• The Geometry of Deep Generative Image Models and its Applications [0.0]
  Generative adversarial networks (GANs) have emerged as a powerful unsupervised method to model the statistical patterns of real-world data sets. These networks are trained to map random inputs in their latent space to new samples representative of the learned data. The structure of the latent space is hard to intuit due to its high dimensionality and the non-linearity of the generator.
  arXiv  Detail & Related papers  (2021-01-15T07:57:33Z)
• Manifold Learning via Manifold Deflation [105.7418091051558]
  dimensionality reduction methods provide a valuable means to visualize and interpret high-dimensional data. Many popular methods can fail dramatically, even on simple two-dimensional Manifolds. This paper presents an embedding method for a novel, incremental tangent space estimator that incorporates global structure as coordinates. Empirically, we show our algorithm recovers novel and interesting embeddings on real-world and synthetic datasets.
  arXiv  Detail & Related papers  (2020-07-07T10:04:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.