Interpreting the Latent Space of GANs via Correlation Analysis for Controllable Concept Manipulation

• URL: http://arxiv.org/abs/2006.10132v2
• Date: Thu, 23 Jul 2020 15:07:58 GMT
• Title: Interpreting the Latent Space of GANs via Correlation Analysis for Controllable Concept Manipulation
• Authors: Ziqiang Li, Rentuo Tao, Hongjing Niu, Bin Li
• Abstract summary: Generative adversarial nets (GANs) have been successfully applied in many fields like image generation, inpainting, super-resolution and drug discovery. This paper proposes a method for interpreting the latent space of GANs by analyzing the correlation between latent variables and the corresponding semantic contents in generated images.
• Score: 9.207806788490057
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative adversarial nets (GANs) have been successfully applied in many fields like image generation, inpainting, super-resolution and drug discovery, etc., by now, the inner process of GANs is far from been understood. To get deeper insight of the intrinsic mechanism of GANs, in this paper, a method for interpreting the latent space of GANs by analyzing the correlation between latent variables and the corresponding semantic contents in generated images is proposed. Unlike previous methods that focus on dissecting models via feature visualization, the emphasis of this work is put on the variables in latent space, i.e. how the latent variables affect the quantitative analysis of generated results. Given a pretrained GAN model with weights fixed, the latent variables are intervened to analyze their effect on the semantic content in generated images. A set of controlling latent variables can be derived for specific content generation, and the controllable semantic content manipulation be achieved. The proposed method is testified on the datasets Fashion-MNIST and UT Zappos50K, experiment results show its effectiveness.

Related papers

• Learning In-between Imagery Dynamics via Physical Latent Spaces [0.7366405857677226]
  We present a framework designed to learn the underlying dynamics between two images observed at consecutive time steps. By incorporating a latent variable that follows a physical model expressed in partial differential equations (PDEs), our approach ensures the interpretability of the learned model. We demonstrate the robustness and effectiveness of our learning framework through a series of numerical tests using geoscientific imagery data.
  arXiv  Detail & Related papers  (2023-10-14T05:14:51Z)
• Discovering Class-Specific GAN Controls for Semantic Image Synthesis [73.91655061467988]
  We propose a novel method for finding spatially disentangled class-specific directions in the latent space of pretrained SIS models. We show that the latent directions found by our method can effectively control the local appearance of semantic classes.
  arXiv  Detail & Related papers  (2022-12-02T21:39:26Z)
• Learning Conditional Invariance through Cycle Consistency [60.85059977904014]
  We propose a novel approach to identify meaningful and independent factors of variation in a dataset. Our method involves two separate latent subspaces for the target property and the remaining input information. We demonstrate on synthetic and molecular data that our approach identifies more meaningful factors which lead to sparser and more interpretable models.
  arXiv  Detail & Related papers  (2021-11-25T17:33:12Z)
• Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style [32.20957709045773]
  We formulate the augmentation process as a latent variable model. We study the identifiability of the latent representation based on pairs of views of the observations. We introduce Causal3DIdent, a dataset of high-dimensional, visually complex images with rich causal dependencies.
  arXiv  Detail & Related papers  (2021-06-08T18:18:09Z)
• Learning Exponential Family Graphical Models with Latent Variables using Regularized Conditional Likelihood [10.21814909876358]
  We present a new convex relaxation framework based on regularized conditional likelihood for latent-variable graphical modeling. We demonstrate the utility and flexibility of our framework via a series of numerical experiments on synthetic as well as real data.
  arXiv  Detail & Related papers  (2020-10-19T11:16:26Z)
• Learning Disentangled Representations with Latent Variation Predictability [102.4163768995288]
  This paper defines the variation predictability of latent disentangled representations. Within an adversarial generation process, we encourage variation predictability by maximizing the mutual information between latent variations and corresponding image pairs. We develop an evaluation metric that does not rely on the ground-truth generative factors to measure the disentanglement of latent representations.
  arXiv  Detail & Related papers  (2020-07-25T08:54:26Z)
• Unsupervised Controllable Generation with Self-Training [90.04287577605723]
  controllable generation with GANs remains a challenging research problem. We propose an unsupervised framework to learn a distribution of latent codes that control the generator through self-training. Our framework exhibits better disentanglement compared to other variants such as the variational autoencoder.
  arXiv  Detail & Related papers  (2020-07-17T21:50:35Z)
• Closed-Form Factorization of Latent Semantics in GANs [65.42778970898534]
  A rich set of interpretable dimensions has been shown to emerge in the latent space of the Generative Adversarial Networks (GANs) trained for synthesizing images. In this work, we examine the internal representation learned by GANs to reveal the underlying variation factors in an unsupervised manner. We propose a closed-form factorization algorithm for latent semantic discovery by directly decomposing the pre-trained weights.
  arXiv  Detail & Related papers  (2020-07-13T18:05:36Z)
• InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs [73.27299786083424]
  We propose a framework called InterFaceGAN to interpret the disentangled face representation learned by state-of-the-art GAN models. We first find that GANs learn various semantics in some linear subspaces of the latent space. We then conduct a detailed study on the correlation between different semantics and manage to better disentangle them via subspace projection.
  arXiv  Detail & Related papers  (2020-05-18T18:01:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.