EGC: Image Generation and Classification via a Diffusion Energy-Based Model

• URL: http://arxiv.org/abs/2304.02012v3
• Date: Thu, 13 Apr 2023 12:24:12 GMT
• Title: EGC: Image Generation and Classification via a Diffusion Energy-Based Model
• Authors: Qiushan Guo, Chuofan Ma, Yi Jiang, Zehuan Yuan, Yizhou Yu, Ping Luo
• Abstract summary: This work introduces an energy-based classifier and generator, namely EGC, which can achieve superior performance in both tasks using a single neural network. EGC achieves competitive generation results compared with state-of-the-art approaches on ImageNet-1k, CelebA-HQ and LSUN Church. This work represents the first successful attempt to simultaneously excel in both tasks using a single set of network parameters.
• Score: 59.591755258395594
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning image classification and image generation using the same set of network parameters is a challenging problem. Recent advanced approaches perform well in one task often exhibit poor performance in the other. This work introduces an energy-based classifier and generator, namely EGC, which can achieve superior performance in both tasks using a single neural network. Unlike a conventional classifier that outputs a label given an image (i.e., a conditional distribution $p(y|\mathbf{x})$), the forward pass in EGC is a classifier that outputs a joint distribution $p(\mathbf{x},y)$, enabling an image generator in its backward pass by marginalizing out the label $y$. This is done by estimating the energy and classification probability given a noisy image in the forward pass, while denoising it using the score function estimated in the backward pass. EGC achieves competitive generation results compared with state-of-the-art approaches on ImageNet-1k, CelebA-HQ and LSUN Church, while achieving superior classification accuracy and robustness against adversarial attacks on CIFAR-10. This work represents the first successful attempt to simultaneously excel in both tasks using a single set of network parameters. We believe that EGC bridges the gap between discriminative and generative learning.

Related papers

• Any Target Can be Offense: Adversarial Example Generation via Generalized Latent Infection [83.72430401516674]
  GAKer is able to construct adversarial examples to any target class. Our method achieves an approximately $14.13%$ higher attack success rate for unknown classes.
  arXiv  Detail & Related papers  (2024-07-17T03:24:09Z)
• Traditional Classification Neural Networks are Good Generators: They are Competitive with DDPMs and GANs [104.72108627191041]
  We show that conventional neural network classifiers can generate high-quality images comparable to state-of-the-art generative models. We propose a mask-based reconstruction module to make semantic gradients-aware to synthesize plausible images. We show that our method is also applicable to text-to-image generation by regarding image-text foundation models.
  arXiv  Detail & Related papers  (2022-11-27T11:25:35Z)
• Rethinking the Paradigm of Content Constraints in Unpaired Image-to-Image Translation [9.900050049833986]
  We propose EnCo, a simple but efficient way to maintain the content by constraining the representational similarity in the latent space of patch-level features. For the similarity function, we use a simple MSE loss instead of contrastive loss, which is currently widely used in I2I tasks. In addition, we rethink the role played by discriminators in sampling patches and propose a discnative attention-guided (DAG) patch sampling strategy to replace random sampling.
  arXiv  Detail & Related papers  (2022-11-20T04:39:57Z)
• Generative Convolution Layer for Image Generation [8.680676599607125]
  This paper introduces a novel convolution method, called generative convolution (GConv) GConv first selects useful kernels compatible with the given latent vector, and then linearly combines the selected kernels to make latent-specific kernels. Using the latent-specific kernels, the proposed method produces the latent-specific features which encourage the generator to produce high-quality images.
  arXiv  Detail & Related papers  (2021-11-30T07:14:12Z)
• Dual Projection Generative Adversarial Networks for Conditional Image Generation [26.563829113916942]
  We propose a Dual Projection GAN (P2GAN) model that learns to balance between em data matching and em label matching. We then propose an improved cGAN model with Auxiliary Classification that directly aligns the fake and real conditionals $P(textclass|textimage)$ by minimizing their $f$-divergence.
  arXiv  Detail & Related papers  (2021-08-20T06:10:38Z)
• Guiding GANs: How to control non-conditional pre-trained GANs for conditional image generation [69.10717733870575]
  We present a novel method for guiding generic non-conditional GANs to behave as conditional GANs. Our approach adds into the mix an encoder network to generate the high-dimensional random input that are fed to the generator network of a non-conditional GAN.
  arXiv  Detail & Related papers  (2021-01-04T14:03:32Z)
• Sequential Graph Convolutional Network for Active Learning [53.99104862192055]
  We propose a novel pool-based Active Learning framework constructed on a sequential Graph Convolution Network (GCN) With a small number of randomly sampled images as seed labelled examples, we learn the parameters of the graph to distinguish labelled vs unlabelled nodes. We exploit these characteristics of GCN to select the unlabelled examples which are sufficiently different from labelled ones.
  arXiv  Detail & Related papers  (2020-06-18T00:55:10Z)
• GATCluster: Self-Supervised Gaussian-Attention Network for Image Clustering [9.722607434532883]
  We propose a self-supervised clustering network for image Clustering (GATCluster) Rather than extracting intermediate features first and then performing the traditional clustering, GATCluster semantic cluster labels without further post-processing. We develop a two-step learning algorithm that is memory-efficient for clustering large-size images.
  arXiv  Detail & Related papers  (2020-02-27T00:57:18Z)
• OneGAN: Simultaneous Unsupervised Learning of Conditional Image Generation, Foreground Segmentation, and Fine-Grained Clustering [100.32273175423146]
  We present a method for simultaneously learning, in an unsupervised manner, a conditional image generator, foreground extraction and segmentation, and object removal and background completion. The method combines a Geneversarative Adrial Network and a Variational Auto-Encoder, with multiple encoders, generators and discriminators, and benefits from solving all tasks at once.
  arXiv  Detail & Related papers  (2019-12-31T18:15:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.