Distilling portable Generative Adversarial Networks for Image Translation

• URL: http://arxiv.org/abs/2003.03519v1
• Date: Sat, 7 Mar 2020 05:53:01 GMT
• Title: Distilling portable Generative Adversarial Networks for Image Translation
• Authors: Hanting Chen, Yunhe Wang, Han Shu, Changyuan Wen, Chunjing Xu, Boxin Shi, Chao Xu, Chang Xu
• Abstract summary: Traditional network compression methods focus on visually recognition tasks, but never deal with generation tasks. Inspired by knowledge distillation, a student generator of fewer parameters is trained by inheriting the low-level and high-level information from the original heavy teacher generator. An adversarial learning process is established to optimize student generator and student discriminator.
• Score: 101.33731583985902
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite Generative Adversarial Networks (GANs) have been widely used in various image-to-image translation tasks, they can be hardly applied on mobile devices due to their heavy computation and storage cost. Traditional network compression methods focus on visually recognition tasks, but never deal with generation tasks. Inspired by knowledge distillation, a student generator of fewer parameters is trained by inheriting the low-level and high-level information from the original heavy teacher generator. To promote the capability of student generator, we include a student discriminator to measure the distances between real images, and images generated by student and teacher generators. An adversarial learning process is therefore established to optimize student generator and student discriminator. Qualitative and quantitative analysis by conducting experiments on benchmark datasets demonstrate that the proposed method can learn portable generative models with strong performance.

Related papers

• Efficient Visualization of Neural Networks with Generative Models and Adversarial Perturbations [0.0]
  This paper presents a novel approach for deep visualization via a generative network, offering an improvement over existing methods. Our model simplifies the architecture by reducing the number of networks used, requiring only a generator and a discriminator. Our model requires less prior training knowledge and uses a non-adversarial training process, where the discriminator acts as a guide.
  arXiv  Detail & Related papers  (2024-09-20T14:59:25Z)
• Generator Born from Classifier [66.56001246096002]
  We aim to reconstruct an image generator, without relying on any data samples. We propose a novel learning paradigm, in which the generator is trained to ensure that the convergence conditions of the network parameters are satisfied.
  arXiv  Detail & Related papers  (2023-12-05T03:41:17Z)
• End to End Generative Meta Curriculum Learning For Medical Data Augmentation [2.471925498075058]
  Current medical image synthetic augmentation techniques rely on intensive use of generative adversarial networks (GANs) We introduce a novel generative meta curriculum learning method that trains the task-specific model (student) end-to-end with only one additional teacher model. In contrast to the generator and discriminator in GAN, which compete with each other, the teacher and student collaborate to improve the student's performance on the target tasks.
  arXiv  Detail & Related papers  (2022-12-20T08:54:11Z)
• Distilling Representations from GAN Generator via Squeeze and Span [55.76208869775715]
  We propose to distill knowledge from GAN generators by squeezing and spanning their representations. We span the distilled representation of the synthetic domain to the real domain by also using real training data to remedy the mode collapse of GANs.
  arXiv  Detail & Related papers  (2022-11-06T01:10:28Z)
• Momentum Adversarial Distillation: Handling Large Distribution Shifts in Data-Free Knowledge Distillation [65.28708064066764]
  We propose a simple yet effective method called Momentum Adversarial Distillation (MAD) MAD maintains an exponential moving average (EMA) copy of the generator and uses synthetic samples from both the generator and the EMA generator to train the student. Our experiments on six benchmark datasets including big datasets like ImageNet and Places365 demonstrate the superior performance of MAD over competing methods.
  arXiv  Detail & Related papers  (2022-09-21T13:53:56Z)
• Generative Cooperative Networks for Natural Language Generation [25.090455367573988]
  We introduce Generative Cooperative Networks, in which the discriminator architecture is cooperatively used along with the generation policy to output samples of realistic texts. We give theoretical guarantees of convergence for our approach, and study various efficient decoding schemes to empirically achieve state-of-the-art results in two main NLG tasks.
  arXiv  Detail & Related papers  (2022-01-28T18:36:57Z)
• Discriminative-Generative Representation Learning for One-Class Anomaly Detection [22.500931323372303]
  We propose a self-supervised learning framework combining generative methods and discriminative methods. Our method significantly outperforms several state-of-the-arts on multiple benchmark data sets.
  arXiv  Detail & Related papers  (2021-07-27T11:46:15Z)
• Representation Consolidation for Training Expert Students [54.90754502493968]
  We show that a multi-head, multi-task distillation method is sufficient to consolidate representations from task-specific teacher(s) and improve downstream performance. Our method can also combine the representational knowledge of multiple teachers trained on one or multiple domains into a single model.
  arXiv  Detail & Related papers  (2021-07-16T17:58:18Z)
• Teachers Do More Than Teach: Compressing Image-to-Image Models [35.40756344110666]
  Generative Adversarial Networks (GANs) have achieved huge success in generating high-fidelity images. GANs suffer from low efficiency due to tremendous computational cost and bulky memory usage. Recent efforts on compression GANs show noticeable progress in obtaining smaller generators.
  arXiv  Detail & Related papers  (2021-03-05T04:29:34Z)
• Generative Hierarchical Features from Synthesizing Images [65.66756821069124]
  We show that learning to synthesize images can bring remarkable hierarchical visual features that are generalizable across a wide range of applications. The visual feature produced by our encoder, termed as Generative Hierarchical Feature (GH-Feat), has strong transferability to both generative and discriminative tasks.
  arXiv  Detail & Related papers  (2020-07-20T18:04:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.