LARGE: Latent-Based Regression through GAN Semantics
- URL: http://arxiv.org/abs/2107.11186v1
- Date: Thu, 22 Jul 2021 17:55:35 GMT
- Title: LARGE: Latent-Based Regression through GAN Semantics
- Authors: Yotam Nitzan, Rinon Gal, Ofir Brenner, Daniel Cohen-Or
- Abstract summary: We propose a novel method for solving regression tasks using few-shot or weak supervision.
We show that our method can be applied across a wide range of domains, leverage multiple latent direction discovery frameworks, and achieve state-of-the-art results.
- Score: 42.50535188836529
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We propose a novel method for solving regression tasks using few-shot or weak
supervision. At the core of our method is the fundamental observation that GANs
are incredibly successful at encoding semantic information within their latent
space, even in a completely unsupervised setting. For modern generative
frameworks, this semantic encoding manifests as smooth, linear directions which
affect image attributes in a disentangled manner. These directions have been
widely used in GAN-based image editing. We show that such directions are not
only linear, but that the magnitude of change induced on the respective
attribute is approximately linear with respect to the distance traveled along
them. By leveraging this observation, our method turns a pre-trained GAN into a
regression model, using as few as two labeled samples. This enables solving
regression tasks on datasets and attributes which are difficult to produce
quality supervision for. Additionally, we show that the same latent-distances
can be used to sort collections of images by the strength of given attributes,
even in the absence of explicit supervision. Extensive experimental evaluations
demonstrate that our method can be applied across a wide range of domains,
leverage multiple latent direction discovery frameworks, and achieve
state-of-the-art results in few-shot and low-supervision settings, even when
compared to methods designed to tackle a single task.
Related papers
- SC2GAN: Rethinking Entanglement by Self-correcting Correlated GAN Space [16.040942072859075]
Gene Networks that achieve following editing directions for one attribute could result in entangled changes with other attributes.
We propose a novel framework SC$2$GAN disentanglement by re-projecting low-density latent code samples in the original latent space.
arXiv Detail & Related papers (2023-10-10T14:42:32Z) - Evaluating the Label Efficiency of Contrastive Self-Supervised Learning
for Multi-Resolution Satellite Imagery [0.0]
Self-supervised learning has been applied in the remote sensing domain to exploit readily-available unlabeled data.
In this paper, we study self-supervised visual representation learning through the lens of label efficiency.
arXiv Detail & Related papers (2022-10-13T06:54:13Z) - Exploring Gradient-based Multi-directional Controls in GANs [19.950198707910587]
We propose a novel approach that discovers nonlinear controls, which enables multi-directional manipulation as well as effective disentanglement.
Our approach is able to gain fine-grained controls over a diverse set of bi-directional and multi-directional attributes, and we showcase its ability to achieve disentanglement significantly better than state-of-the-art methods.
arXiv Detail & Related papers (2022-09-01T19:10:26Z) - Activation to Saliency: Forming High-Quality Labels for Unsupervised
Salient Object Detection [54.92703325989853]
We propose a two-stage Activation-to-Saliency (A2S) framework that effectively generates high-quality saliency cues.
No human annotations are involved in our framework during the whole training process.
Our framework reports significant performance compared with existing USOD methods.
arXiv Detail & Related papers (2021-12-07T11:54:06Z) - Latent Transformations via NeuralODEs for GAN-based Image Editing [25.272389610447856]
We show that nonlinear latent code manipulations realized as flows of a trainable Neural ODE are beneficial for many practical non-face image domains.
In particular, we investigate a large number of datasets with known attributes and demonstrate that certain attribute manipulations are challenging to obtain with linear shifts only.
arXiv Detail & Related papers (2021-11-29T18:59:54Z) - Orthogonal Jacobian Regularization for Unsupervised Disentanglement in
Image Generation [64.92152574895111]
We propose a simple Orthogonal Jacobian Regularization (OroJaR) to encourage deep generative model to learn disentangled representations.
Our method is effective in disentangled and controllable image generation, and performs favorably against the state-of-the-art methods.
arXiv Detail & Related papers (2021-08-17T15:01:46Z) - Unsupervised Discovery of Disentangled Manifolds in GANs [74.24771216154105]
Interpretable generation process is beneficial to various image editing applications.
We propose a framework to discover interpretable directions in the latent space given arbitrary pre-trained generative adversarial networks.
arXiv Detail & Related papers (2020-11-24T02:18:08Z) - Unsupervised Controllable Generation with Self-Training [90.04287577605723]
controllable generation with GANs remains a challenging research problem.
We propose an unsupervised framework to learn a distribution of latent codes that control the generator through self-training.
Our framework exhibits better disentanglement compared to other variants such as the variational autoencoder.
arXiv Detail & Related papers (2020-07-17T21:50:35Z) - Closed-Form Factorization of Latent Semantics in GANs [65.42778970898534]
A rich set of interpretable dimensions has been shown to emerge in the latent space of the Generative Adversarial Networks (GANs) trained for synthesizing images.
In this work, we examine the internal representation learned by GANs to reveal the underlying variation factors in an unsupervised manner.
We propose a closed-form factorization algorithm for latent semantic discovery by directly decomposing the pre-trained weights.
arXiv Detail & Related papers (2020-07-13T18:05:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.