Interpreting Galaxy Deblender GAN from the Discriminator's Perspective
- URL: http://arxiv.org/abs/2001.06151v1
- Date: Fri, 17 Jan 2020 04:05:46 GMT
- Title: Interpreting Galaxy Deblender GAN from the Discriminator's Perspective
- Authors: Heyi Li, Yuewei Lin, Klaus Mueller, Wei Xu
- Abstract summary: This research focuses on behaviors of one of the network's major components, the Discriminator, which plays a vital role but is often overlooked.
We demonstrate that our method clearly reveals attention areas of the Discriminator when differentiating generated galaxy images from ground truth images.
- Score: 50.12901802952574
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Generative adversarial networks (GANs) are well known for their unsupervised
learning capabilities. A recent success in the field of astronomy is deblending
two overlapping galaxy images via a branched GAN model. However, it remains a
significant challenge to comprehend how the network works, which is
particularly difficult for non-expert users. This research focuses on behaviors
of one of the network's major components, the Discriminator, which plays a
vital role but is often overlooked, Specifically, we enhance the Layer-wise
Relevance Propagation (LRP) scheme to generate a heatmap-based visualization.
We call this technique Polarized-LRP and it consists of two parts i.e. positive
contribution heatmaps for ground truth images and negative contribution
heatmaps for generated images. Using the Galaxy Zoo dataset we demonstrate that
our method clearly reveals attention areas of the Discriminator when
differentiating generated galaxy images from ground truth images. To connect
the Discriminator's impact on the Generator, we visualize the gradual changes
of the Generator across the training process. An interesting result we have
achieved there is the detection of a problematic data augmentation procedure
that would else have remained hidden. We find that our proposed method serves
as a useful visual analytical tool for a deeper understanding of GAN models.
Related papers
- Contrasting Deepfakes Diffusion via Contrastive Learning and Global-Local Similarities [88.398085358514]
Contrastive Deepfake Embeddings (CoDE) is a novel embedding space specifically designed for deepfake detection.
CoDE is trained via contrastive learning by additionally enforcing global-local similarities.
arXiv Detail & Related papers (2024-07-29T18:00:10Z) - GenFace: A Large-Scale Fine-Grained Face Forgery Benchmark and Cross Appearance-Edge Learning [50.7702397913573]
The rapid advancement of photorealistic generators has reached a critical juncture where the discrepancy between authentic and manipulated images is increasingly indistinguishable.
Although there have been a number of publicly available face forgery datasets, the forgery faces are mostly generated using GAN-based synthesis technology.
We propose a large-scale, diverse, and fine-grained high-fidelity dataset, namely GenFace, to facilitate the advancement of deepfake detection.
arXiv Detail & Related papers (2024-02-03T03:13:50Z) - X-Transfer: A Transfer Learning-Based Framework for GAN-Generated Fake
Image Detection [33.31312811230408]
misuse of GANs for generating deceptive images, such as face replacement, raises significant security concerns.
This paper introduces a novel GAN-generated image detection algorithm called X-Transfer.
It enhances transfer learning by utilizing two neural networks that employ interleaved parallel gradient transmission.
arXiv Detail & Related papers (2023-10-07T01:23:49Z) - An Attention-Guided and Wavelet-Constrained Generative Adversarial
Network for Infrared and Visible Image Fusion [10.900528467160816]
We propose an attention-guided and wavelet-constrained GAN for infrared and visible image fusion (AWFGAN)
Specifically, we introduce the spatial attention modules (SAM) into the generator to obtain the spatial attention maps.
We extend the discrimination range of visible information to the wavelet subspace, which can force the generator to restore the high-frequency details of visible images.
arXiv Detail & Related papers (2022-10-20T05:01:20Z) - MC-LCR: Multi-modal contrastive classification by locally correlated
representations for effective face forgery detection [11.124150983521158]
We propose a novel framework named Multi-modal Contrastive Classification by Locally Correlated Representations.
Our MC-LCR aims to amplify implicit local discrepancies between authentic and forged faces from both spatial and frequency domains.
We achieve state-of-the-art performance and demonstrate the robustness and generalization of our method.
arXiv Detail & Related papers (2021-10-07T09:24:12Z) - Heterogeneous Face Frontalization via Domain Agnostic Learning [74.86585699909459]
We propose a domain agnostic learning-based generative adversarial network (DAL-GAN) which can synthesize frontal views in the visible domain from thermal faces with pose variations.
DAL-GAN consists of a generator with an auxiliary classifier and two discriminators which capture both local and global texture discriminations for better synthesis.
arXiv Detail & Related papers (2021-07-17T20:41:41Z) - Diamond in the rough: Improving image realism by traversing the GAN
latent space [0.0]
We present an unsupervised method to find a direction in the latent space that aligns with improved photo-realism.
Our approach leaves the network unchanged while enhancing the fidelity of the generated image.
We use a simple generator inversion to find the direction in the latent space that results in the smallest change in the image space.
arXiv Detail & Related papers (2021-04-12T14:45:29Z) - Unsupervised Discovery of Disentangled Manifolds in GANs [74.24771216154105]
Interpretable generation process is beneficial to various image editing applications.
We propose a framework to discover interpretable directions in the latent space given arbitrary pre-trained generative adversarial networks.
arXiv Detail & Related papers (2020-11-24T02:18:08Z) - DeshuffleGAN: A Self-Supervised GAN to Improve Structure Learning [0.0]
We argue that one of the crucial points to improve the GAN performance is to be able to provide the model with a capability to learn the spatial structure in data.
We introduce a deshuffling task that solves a puzzle of randomly shuffled image tiles, which in turn helps the DeshuffleGAN learn to increase its expressive capacity for spatial structure and realistic appearance.
arXiv Detail & Related papers (2020-06-15T19:06:07Z) - InterFaceGAN: Interpreting the Disentangled Face Representation Learned
by GANs [73.27299786083424]
We propose a framework called InterFaceGAN to interpret the disentangled face representation learned by state-of-the-art GAN models.
We first find that GANs learn various semantics in some linear subspaces of the latent space.
We then conduct a detailed study on the correlation between different semantics and manage to better disentangle them via subspace projection.
arXiv Detail & Related papers (2020-05-18T18:01:22Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.