Instance-Conditioned GAN
- URL: http://arxiv.org/abs/2109.05070v1
- Date: Fri, 10 Sep 2021 19:08:45 GMT
- Title: Instance-Conditioned GAN
- Authors: Arantxa Casanova, Marl\`ene Careil, Jakob Verbeek, Michal Drozdzal,
Adriana Romero-Soriano
- Abstract summary: Generative Adversarial Networks (GANs) can generate near photo realistic images in narrow domains such as human faces.
We take inspiration from kernel density estimation techniques and introduce a non-parametric approach to modeling distributions of complex datasets.
- Score: 26.27527697877534
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Generative Adversarial Networks (GANs) can generate near photo realistic
images in narrow domains such as human faces. Yet, modeling complex
distributions of datasets such as ImageNet and COCO-Stuff remains challenging
in unconditional settings. In this paper, we take inspiration from kernel
density estimation techniques and introduce a non-parametric approach to
modeling distributions of complex datasets. We partition the data manifold into
a mixture of overlapping neighborhoods described by a datapoint and its nearest
neighbors, and introduce a model, called instance-conditioned GAN (IC-GAN),
which learns the distribution around each datapoint. Experimental results on
ImageNet and COCO-Stuff show that IC-GAN significantly improves over
unconditional models and unsupervised data partitioning baselines. Moreover, we
show that IC-GAN can effortlessly transfer to datasets not seen during training
by simply changing the conditioning instances, and still generate realistic
images. Finally, we extend IC-GAN to the class-conditional case and show
semantically controllable generation and competitive quantitative results on
ImageNet; while improving over BigGAN on ImageNet-LT. We will opensource our
code and trained models to reproduce the reported results.
Related papers
- Deep Domain Adaptation: A Sim2Real Neural Approach for Improving Eye-Tracking Systems [80.62854148838359]
Eye image segmentation is a critical step in eye tracking that has great influence over the final gaze estimate.
We use dimensionality-reduction techniques to measure the overlap between the target eye images and synthetic training data.
Our methods result in robust, improved performance when tackling the discrepancy between simulation and real-world data samples.
arXiv Detail & Related papers (2024-03-23T22:32:06Z) - Generative adversarial networks for data-scarce spectral applications [0.0]
We report on an application of GANs in the domain of synthetic spectral data generation.
We show that CWGANs can act as a surrogate model with improved performance in the low-data regime.
arXiv Detail & Related papers (2023-07-14T16:27:24Z) - Hierarchical Integration Diffusion Model for Realistic Image Deblurring [71.76410266003917]
Diffusion models (DMs) have been introduced in image deblurring and exhibited promising performance.
We propose the Hierarchical Integration Diffusion Model (HI-Diff), for realistic image deblurring.
Experiments on synthetic and real-world blur datasets demonstrate that our HI-Diff outperforms state-of-the-art methods.
arXiv Detail & Related papers (2023-05-22T12:18:20Z) - LD-GAN: Low-Dimensional Generative Adversarial Network for Spectral
Image Generation with Variance Regularization [72.4394510913927]
Deep learning methods are state-of-the-art for spectral image (SI) computational tasks.
GANs enable diverse augmentation by learning and sampling from the data distribution.
GAN-based SI generation is challenging since the high-dimensionality nature of this kind of data hinders the convergence of the GAN training yielding to suboptimal generation.
We propose a statistical regularization to control the low-dimensional representation variance for the autoencoder training and to achieve high diversity of samples generated with the GAN.
arXiv Detail & Related papers (2023-04-29T00:25:02Z) - Intriguing Property and Counterfactual Explanation of GAN for Remote Sensing Image Generation [25.96740500337747]
Generative adversarial networks (GANs) have achieved remarkable progress in the natural image field.
GAN model is more sensitive to the size of training data for RS image generation than for natural image generation.
We propose two innovative adjustment schemes, namely Uniformity Regularization (UR) and Entropy Regularization (ER), to increase the information learned by the GAN model.
arXiv Detail & Related papers (2023-03-09T13:22:50Z) - Unifying conditional and unconditional semantic image synthesis with
OCO-GAN [29.77186837186815]
We propose OCO-GAN, for Optionally COnditioned GAN, which addresses both tasks in a unified manner.
trained adversarially in an end-to-end approach with a shared discriminator, we are able to leverage the synergy between both tasks.
Our results are competitive or better than state-of-the art specialised unconditional and conditional models.
arXiv Detail & Related papers (2022-11-25T13:43:21Z) - GAN-Supervised Dense Visual Alignment [95.37027391102684]
We propose GAN-Supervised Learning, a framework for learning discriminative models and their GAN-generated training data jointly end-to-end.
Inspired by the classic Congealing method, our GANgealing algorithm trains a Spatial Transformer to map random samples from a GAN trained on unaligned data to a common, jointly-learned target mode.
arXiv Detail & Related papers (2021-12-09T18:59:58Z) - InvGAN: Invertible GANs [88.58338626299837]
InvGAN, short for Invertible GAN, successfully embeds real images to the latent space of a high quality generative model.
This allows us to perform image inpainting, merging, and online data augmentation.
arXiv Detail & Related papers (2021-12-08T21:39:00Z) - Conditional Generation of Synthetic Geospatial Images from Pixel-level
and Feature-level Inputs [0.0]
We present a conditional generative model, called VAE-Info-cGAN, for synthesizing semantically rich images simultaneously conditioned on a pixel-level condition (PLC) and a feature-level condition (FLC)
The proposed model can accurately generate various forms of macroscopic aggregates across different geographic locations while conditioned only on atemporal representation of the road network.
arXiv Detail & Related papers (2021-09-11T06:58:19Z) - VAE-Info-cGAN: Generating Synthetic Images by Combining Pixel-level and
Feature-level Geospatial Conditional Inputs [0.0]
We present a conditional generative model for synthesizing semantically rich images simultaneously conditioned on a pixellevel (PLC) and a featurelevel condition (FLC)
Experiments on a GPS dataset show that the proposed model can accurately generate various forms of macroscopic aggregates across different geographic locations.
arXiv Detail & Related papers (2020-12-08T03:46:19Z) - Improving Generative Adversarial Networks with Local Coordinate Coding [150.24880482480455]
Generative adversarial networks (GANs) have shown remarkable success in generating realistic data from some predefined prior distribution.
In practice, semantic information might be represented by some latent distribution learned from data.
We propose an LCCGAN model with local coordinate coding (LCC) to improve the performance of generating data.
arXiv Detail & Related papers (2020-07-28T09:17:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.