Using Deep LSD to build operators in GANs latent space with meaning in
real space
- URL: http://arxiv.org/abs/2102.05132v1
- Date: Tue, 9 Feb 2021 21:05:20 GMT
- Title: Using Deep LSD to build operators in GANs latent space with meaning in
real space
- Authors: J. Quetzalcoatl Toledo-Marin and James A. Glazier
- Abstract summary: Lack of correlation is important because it suggests that the latent space manifold is simpler to understand and manipulate.
Generative models are widely used in deep learning, e.g., variational autoencoders (VAEs) and generative adversarial networks (GANs)
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Generative models rely on the key idea that data can be represented in terms
of latent variables which are uncorrelated by definition. Lack of correlation
is important because it suggests that the latent space manifold is simpler to
understand and manipulate. Generative models are widely used in deep learning,
e.g., variational autoencoders (VAEs) and generative adversarial networks
(GANs). Here we propose a method to build a set of linearly independent vectors
in the latent space of a GANs, which we call quasi-eigenvectors. These
quasi-eigenvectors have two key properties: i) They span all the latent space,
ii) A set of these quasi-eigenvectors map to each of the labeled features
one-on-one. We show that in the case of the MNIST, while the number of
dimensions in latent space is large by construction, 98% of the data in real
space map to a sub-domain of latent space of dimensionality equal to the number
of labels. We then show how the quasi-eigenvalues can be used for Latent
Spectral Decomposition (LSD), which has applications in denoising images and
for performing matrix operations in latent space that map to feature
transformations in real space. We show how this method provides insight into
the latent space topology. The key point is that the set of quasi-eigenvectors
form a basis set in latent space and each direction corresponds to a feature in
real space.
Related papers
- Interpreting the Weight Space of Customized Diffusion Models [79.14866339932199]
We investigate the space of weights spanned by a large collection of customized diffusion models.
We populate this space by creating a dataset of over 60,000 models, each of which is a base model fine-tuned to insert a different person's visual identity.
We demonstrate three immediate applications of this space -- sampling, editing, and inversion.
arXiv Detail & Related papers (2024-06-13T17:59:56Z) - Transport of Algebraic Structure to Latent Embeddings [8.693845596949892]
Machine learning often aims to produce latent embeddings of inputs which lie in a larger, abstract mathematical space.
How can we learn to "union" two sets using only their latent embeddings while respecting associativity?
We propose a general procedure for parameterizing latent space operations that are provably consistent with the laws on the input space.
arXiv Detail & Related papers (2024-05-27T02:24:57Z) - Householder Projector for Unsupervised Latent Semantics Discovery [58.92485745195358]
Householder Projector helps StyleGANs to discover more disentangled and precise semantic attributes without sacrificing image fidelity.
We integrate our projector into pre-trained StyleGAN2/StyleGAN3 and evaluate the models on several benchmarks.
arXiv Detail & Related papers (2023-07-16T11:43:04Z) - Intrinsic dimension estimation for discrete metrics [65.5438227932088]
In this letter we introduce an algorithm to infer the intrinsic dimension (ID) of datasets embedded in discrete spaces.
We demonstrate its accuracy on benchmark datasets, and we apply it to analyze a metagenomic dataset for species fingerprinting.
This suggests that evolutive pressure acts on a low-dimensional manifold despite the high-dimensionality of sequences' space.
arXiv Detail & Related papers (2022-07-20T06:38:36Z) - Analyzing the Latent Space of GAN through Local Dimension Estimation [4.688163910878411]
style-based GANs (StyleGANs) in high-fidelity image synthesis have motivated research to understand the semantic properties of their latent spaces.
We propose a local dimension estimation algorithm for arbitrary intermediate layers in a pre-trained GAN model.
Our proposed metric, called Distortion, measures an inconsistency of intrinsic space on the learned latent space.
arXiv Detail & Related papers (2022-05-26T06:36:06Z) - Sign and Basis Invariant Networks for Spectral Graph Representation
Learning [75.18802152811539]
We introduce SignNet and BasisNet -- new neural architectures that are invariant to all requisite symmetries and hence process collections of eigenspaces in a principled manner.
Our networks are theoretically strong for graph representation learning -- they can approximate any spectral graph convolution.
Experiments show the strength of our networks for learning spectral graph filters and learning graph positional encodings.
arXiv Detail & Related papers (2022-02-25T23:11:59Z) - EigenGAN: Layer-Wise Eigen-Learning for GANs [84.33920839885619]
EigenGAN is able to unsupervisedly mine interpretable and controllable dimensions from different generator layers.
By traversing the coefficient of a specific eigen-dimension, the generator can produce samples with continuous changes corresponding to a specific semantic attribute.
arXiv Detail & Related papers (2021-04-26T11:14:37Z) - The Geometry of Deep Generative Image Models and its Applications [0.0]
Generative adversarial networks (GANs) have emerged as a powerful unsupervised method to model the statistical patterns of real-world data sets.
These networks are trained to map random inputs in their latent space to new samples representative of the learned data.
The structure of the latent space is hard to intuit due to its high dimensionality and the non-linearity of the generator.
arXiv Detail & Related papers (2021-01-15T07:57:33Z) - Geometry-Aware Hamiltonian Variational Auto-Encoder [0.0]
Variational auto-encoders (VAEs) have proven to be a well suited tool for performing dimensionality reduction by extracting latent variables lying in a potentially much smaller dimensional space than the data.
However, such generative models may perform poorly when trained on small data sets which are abundant in many real-life fields such as medicine.
We argue that such latent space modelling provides useful information about its underlying structure leading to far more meaningfuls, more realistic data-generation and more reliable clustering.
arXiv Detail & Related papers (2020-10-22T08:26:46Z) - Spatial Pyramid Based Graph Reasoning for Semantic Segmentation [67.47159595239798]
We apply graph convolution into the semantic segmentation task and propose an improved Laplacian.
The graph reasoning is directly performed in the original feature space organized as a spatial pyramid.
We achieve comparable performance with advantages in computational and memory overhead.
arXiv Detail & Related papers (2020-03-23T12:28:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.