Interpreting Class Conditional GANs with Channel Awareness
- URL: http://arxiv.org/abs/2203.11173v1
- Date: Mon, 21 Mar 2022 17:53:22 GMT
- Title: Interpreting Class Conditional GANs with Channel Awareness
- Authors: Yingqing He, Zhiyi Zhang, Jiapeng Zhu, Yujun Shen, Qifeng Chen
- Abstract summary: We investigate how a class conditional generator unifies the synthesis of multiple classes.
To describe such a phenomenon, we propose channel awareness, which quantitatively characterizes how a single channel contributes to the final synthesis.
Our algorithm enables several novel applications with conditional GANs.
- Score: 57.01413866290279
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Understanding the mechanism of generative adversarial networks (GANs) helps
us better use GANs for downstream applications. Existing efforts mainly target
interpreting unconditional models, leaving it less explored how a conditional
GAN learns to render images regarding various categories. This work fills in
this gap by investigating how a class conditional generator unifies the
synthesis of multiple classes. For this purpose, we dive into the widely used
class-conditional batch normalization (CCBN), and observe that each feature
channel is activated at varying degrees given different categorical embeddings.
To describe such a phenomenon, we propose channel awareness, which
quantitatively characterizes how a single channel contributes to the final
synthesis. Extensive evaluations and analyses on the BigGAN model pre-trained
on ImageNet reveal that only a subset of channels is primarily responsible for
the generation of a particular category, similar categories (e.g., cat and dog)
usually get related to some same channels, and some channels turn out to share
information across all classes. For good measure, our algorithm enables several
novel applications with conditional GANs. Concretely, we achieve (1) versatile
image editing via simply altering a single channel and manage to (2)
harmoniously hybridize two different classes. We further verify that the
proposed channel awareness shows promising potential in (3) segmenting the
synthesized image and (4) evaluating the category-wise synthesis performance.
Related papers
- A Unified View of cGANs with and without Classifiers [24.28407308818025]
Conditional Generative Adversarial Networks (cGANs) are implicit generative models which allow to sample from class-conditional distributions.
Some representative cGANs avoid the shortcoming and reach state-of-the-art performance without having classifiers.
In this work, we demonstrate that classifiers can be properly leveraged to improve cGANs.
arXiv Detail & Related papers (2021-11-01T15:36:33Z) - Joint Inductive and Transductive Learning for Video Object Segmentation [107.32760625159301]
Semi-supervised object segmentation is a task of segmenting the target object in a video sequence given only a mask in the first frame.
Most previous best-performing methods adopt matching-based transductive reasoning or online inductive learning.
We propose to integrate transductive and inductive learning into a unified framework to exploit complement between them for accurate and robust video object segmentation.
arXiv Detail & Related papers (2021-08-08T16:25:48Z) - GAN for Vision, KG for Relation: a Two-stage Deep Network for Zero-shot
Action Recognition [33.23662792742078]
We propose a two-stage deep neural network for zero-shot action recognition.
In the sampling stage, we utilize a generative adversarial networks (GAN) trained by action features and word vectors of seen classes.
In the classification stage, we construct a knowledge graph based on the relationship between word vectors of action classes and related objects.
arXiv Detail & Related papers (2021-05-25T09:34:42Z) - Generative Multi-Label Zero-Shot Learning [136.17594611722285]
Multi-label zero-shot learning strives to classify images into multiple unseen categories for which no data is available during training.
Our work is the first to tackle the problem of multi-label feature in the (generalized) zero-shot setting.
Our cross-level fusion-based generative approach outperforms the state-of-the-art on all three datasets.
arXiv Detail & Related papers (2021-01-27T18:56:46Z) - Progressive Co-Attention Network for Fine-grained Visual Classification [20.838908090777885]
Fine-grained visual classification aims to recognize images belonging to multiple sub-categories within a same category.
Most existing methods only take individual image as input.
We propose an effective method called progressive co-attention network (PCA-Net) to tackle this problem.
arXiv Detail & Related papers (2021-01-21T10:19:02Z) - Channel-wise Knowledge Distillation for Dense Prediction [73.99057249472735]
We propose to align features channel-wise between the student and teacher networks.
We consistently achieve superior performance on three benchmarks with various network structures.
arXiv Detail & Related papers (2020-11-26T12:00:38Z) - Symbiotic Adversarial Learning for Attribute-based Person Search [86.7506832053208]
We present a symbiotic adversarial learning framework, called SAL.Two GANs sit at the base of the framework in a symbiotic learning scheme.
Specifically, two different types of generative adversarial networks learn collaboratively throughout the training process.
arXiv Detail & Related papers (2020-07-19T07:24:45Z) - Channel Interaction Networks for Fine-Grained Image Categorization [61.095320862647476]
Fine-grained image categorization is challenging due to the subtle inter-class differences.
We propose a channel interaction network (CIN), which models the channel-wise interplay both within an image and across images.
Our model can be trained efficiently in an end-to-end fashion without the need of multi-stage training and testing.
arXiv Detail & Related papers (2020-03-11T11:51:51Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.