FA-GAN: Feature-Aware GAN for Text to Image Synthesis
- URL: http://arxiv.org/abs/2109.00907v1
- Date: Thu, 2 Sep 2021 13:05:36 GMT
- Title: FA-GAN: Feature-Aware GAN for Text to Image Synthesis
- Authors: Eunyeong Jeon, Kunhee Kim, Daijin Kim
- Abstract summary: We propose a Generative Adversarial Network (GAN) to synthesize a high-quality image by integrating two techniques.
First, we design a self-supervised discriminator with an auxiliary decoder so that the discriminator can extract better representation.
Secondly, we introduce a feature-aware loss to provide the generator more direct supervision by employing the feature representation from the self-supervised discriminator.
- Score: 7.0168039268464
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Text-to-image synthesis aims to generate a photo-realistic image from a given
natural language description. Previous works have made significant progress
with Generative Adversarial Networks (GANs). Nonetheless, it is still hard to
generate intact objects or clear textures (Fig 1). To address this issue, we
propose Feature-Aware Generative Adversarial Network (FA-GAN) to synthesize a
high-quality image by integrating two techniques: a self-supervised
discriminator and a feature-aware loss. First, we design a self-supervised
discriminator with an auxiliary decoder so that the discriminator can extract
better representation. Secondly, we introduce a feature-aware loss to provide
the generator more direct supervision by employing the feature representation
from the self-supervised discriminator. Experiments on the MS-COCO dataset show
that our proposed method significantly advances the state-of-the-art FID score
from 28.92 to 24.58.
Related papers
- Wavelet-based Unsupervised Label-to-Image Translation [9.339522647331334]
We propose a new Unsupervised paradigm for SIS (USIS) that makes use of a self-supervised segmentation loss and whole image wavelet based discrimination.
We test our methodology on 3 challenging datasets and demonstrate its ability to bridge the performance gap between paired and unpaired models.
arXiv Detail & Related papers (2023-05-16T17:48:44Z) - IR-GAN: Image Manipulation with Linguistic Instruction by Increment
Reasoning [110.7118381246156]
Increment Reasoning Generative Adversarial Network (IR-GAN) aims to reason consistency between visual increment in images and semantic increment in instructions.
First, we introduce the word-level and instruction-level instruction encoders to learn user's intention from history-correlated instructions as semantic increment.
Second, we embed the representation of semantic increment into that of source image for generating target image, where source image plays the role of referring auxiliary.
arXiv Detail & Related papers (2022-04-02T07:48:39Z) - A Novel Generator with Auxiliary Branch for Improving GAN Performance [7.005458308454871]
This brief introduces a novel generator architecture that produces the image by combining features obtained through two different branches.
The goal of the main branch is to produce the image by passing through the multiple residual blocks, whereas the auxiliary branch is to convey the coarse information in the earlier layer to the later one.
To prove the superiority of the proposed method, this brief provides extensive experiments using various standard datasets.
arXiv Detail & Related papers (2021-12-30T08:38:49Z) - Semantic-Aware Generation for Self-Supervised Visual Representation
Learning [116.5814634936371]
We advocate for Semantic-aware Generation (SaGe) to facilitate richer semantics rather than details to be preserved in the generated image.
SaGe complements the target network with view-specific features and thus alleviates the semantic degradation brought by intensive data augmentations.
We execute SaGe on ImageNet-1K and evaluate the pre-trained models on five downstream tasks including nearest neighbor test, linear classification, and fine-scaled image recognition.
arXiv Detail & Related papers (2021-11-25T16:46:13Z) - Dual Contrastive Loss and Attention for GANs [82.713118646294]
We propose a novel dual contrastive loss and show that, with this loss, discriminator learns more generalized and distinguishable representations to incentivize generation.
We find attention to be still an important module for successful image generation even though it was not used in the recent state-of-the-art models.
By combining the strengths of these remedies, we improve the compelling state-of-the-art Fr'echet Inception Distance (FID) by at least 17.5% on several benchmark datasets.
arXiv Detail & Related papers (2021-03-31T01:10:26Z) - You Only Need Adversarial Supervision for Semantic Image Synthesis [84.83711654797342]
We propose a novel, simplified GAN model, which needs only adversarial supervision to achieve high quality results.
We show that images synthesized by our model are more diverse and follow the color and texture of real images more closely.
arXiv Detail & Related papers (2020-12-08T23:00:48Z) - DF-GAN: A Simple and Effective Baseline for Text-to-Image Synthesis [80.54273334640285]
We propose a novel one-stage text-to-image backbone that directly synthesizes high-resolution images without entanglements between different generators.
We also propose a novel Target-Aware Discriminator composed of Matching-Aware Gradient Penalty and One-Way Output.
Compared with current state-of-the-art methods, our proposed DF-GAN is simpler but more efficient to synthesize realistic and text-matching images.
arXiv Detail & Related papers (2020-08-13T12:51:17Z) - A U-Net Based Discriminator for Generative Adversarial Networks [86.67102929147592]
We propose an alternative U-Net based discriminator architecture for generative adversarial networks (GANs)
The proposed architecture allows to provide detailed per-pixel feedback to the generator while maintaining the global coherence of synthesized images.
The novel discriminator improves over the state of the art in terms of the standard distribution and image quality metrics.
arXiv Detail & Related papers (2020-02-28T11:16:54Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.