Wavelet Knowledge Distillation: Towards Efficient Image-to-Image Translation

• URL: http://arxiv.org/abs/2203.06321v1
• Date: Sat, 12 Mar 2022 02:42:04 GMT
• Title: Wavelet Knowledge Distillation: Towards Efficient Image-to-Image Translation
• Abstract summary: This paper investigates GANs performance from a frequency perspective. The results show that GANs, especially small GANs lack the ability to generate high-quality high frequency information. We propose a novel knowledge distillation method referred to as wavelet knowledge distillation.
• Score: 22.841122397104265
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Remarkable achievements have been attained with Generative Adversarial Networks (GANs) in image-to-image translation. However, due to a tremendous amount of parameters, state-of-the-art GANs usually suffer from low efficiency and bulky memory usage. To tackle this challenge, firstly, this paper investigates GANs performance from a frequency perspective. The results show that GANs, especially small GANs lack the ability to generate high-quality high frequency information. To address this problem, we propose a novel knowledge distillation method referred to as wavelet knowledge distillation. Instead of directly distilling the generated images of teachers, wavelet knowledge distillation first decomposes the images into different frequency bands with discrete wavelet transformation and then only distills the high frequency bands. As a result, the student GAN can pay more attention to its learning on high frequency bands. Experiments demonstrate that our method leads to 7.08 times compression and 6.80 times acceleration on CycleGAN with almost no performance drop. Additionally, we have studied the relation between discriminators and generators which shows that the compression of discriminators can promote the performance of compressed generators.

Related papers

• One Step Diffusion-based Super-Resolution with Time-Aware Distillation [60.262651082672235]
  Diffusion-based image super-resolution (SR) methods have shown promise in reconstructing high-resolution images with fine details from low-resolution counterparts. Recent techniques have been devised to enhance the sampling efficiency of diffusion-based SR models via knowledge distillation. We propose a time-aware diffusion distillation method, named TAD-SR, to accomplish effective and efficient image super-resolution.
  arXiv  Detail & Related papers  (2024-08-14T11:47:22Z)
• Frequency-Aware Deepfake Detection: Improving Generalizability through Frequency Space Learning [81.98675881423131]
  This research addresses the challenge of developing a universal deepfake detector that can effectively identify unseen deepfake images. Existing frequency-based paradigms have relied on frequency-level artifacts introduced during the up-sampling in GAN pipelines to detect forgeries. We introduce a novel frequency-aware approach called FreqNet, centered around frequency domain learning, specifically designed to enhance the generalizability of deepfake detectors.
  arXiv  Detail & Related papers  (2024-03-12T01:28:00Z)
• FreGS: 3D Gaussian Splatting with Progressive Frequency Regularization [67.47895278233717]
  We develop a progressive frequency regularization technique to tackle the over-reconstruction issue within the frequency space. FreGS achieves superior novel view synthesis and outperforms the state-of-the-art consistently.
  arXiv  Detail & Related papers  (2024-03-11T17:00:27Z)
• FreeKD: Knowledge Distillation via Semantic Frequency Prompt [33.28298466424745]
  We propose a Frequency Knowledge Distillation method as FreeKD, which determines the optimal localization and extent for the frequency distillation. We also employ a position-aware relational frequency loss for dense prediction tasks, delivering a high-order spatial enhancement to the student model.
  arXiv  Detail & Related papers  (2023-11-20T08:06:41Z)
• WaveGAN: Frequency-aware GAN for High-Fidelity Few-shot Image Generation [3.5459430566117893]
  We propose WaveGAN, a frequency-aware model for few-shot image generation. We alleviate the generator's struggles of synthesizing fine details by employing high-frequency skip connections. We achieve new state-of-the-art with FID 42.17, LPIPS 0.3868, FID 30.35, LPIPS 0.5076, and FID 4.96, LPIPS 0.3822 respectively on Flower, Animal Faces, and VGGFace.
  arXiv  Detail & Related papers  (2022-07-15T04:39:47Z)
• Exploring Inter-frequency Guidance of Image for Lightweight Gaussian Denoising [1.52292571922932]
  We propose a novel network architecture denoted as IGNet, in order to refine the frequency bands from low to high in a progressive manner. With this design, more inter-frequency prior and information are utilized, thus the model size can be lightened while still perserves competitive results.
  arXiv  Detail & Related papers  (2021-12-22T10:35:53Z)
• On the Frequency Bias of Generative Models [61.60834513380388]
  We analyze proposed measures against high-frequency artifacts in state-of-the-art GAN training. We find that none of the existing approaches can fully resolve spectral artifacts yet. Our results suggest that there is great potential in improving the discriminator.
  arXiv  Detail & Related papers  (2021-11-03T18:12:11Z)
• Are High-Frequency Components Beneficial for Training of Generative Adversarial Networks [11.226288436817956]
  Generative Adversarial Networks (GANs) have the ability to generate realistic images that are visually indistinguishable from real images. Recent studies of the image spectrum have demonstrated that generated and real images share significant differences at high frequency. We propose two preprocessing methods eliminating high-frequency differences in GANs training.
  arXiv  Detail & Related papers  (2021-03-20T04:37:06Z)
• Noise Homogenization via Multi-Channel Wavelet Filtering for High-Fidelity Sample Generation in GANs [47.92719758687014]
  We propose a novel multi-channel wavelet-based filtering method for Generative Adversarial Networks (GANs) When embedding a wavelet deconvolution layer in the generator, the resultant GAN, called WaveletGAN, takes advantage of the wavelet deconvolution to learn a filtering with multiple channels. We conducted benchmark experiments on the Fashion-MNIST, KMNIST and SVHN datasets through an open GAN benchmark tool.
  arXiv  Detail & Related papers  (2020-05-14T03:40:11Z)
• Blur, Noise, and Compression Robust Generative Adversarial Networks [85.68632778835253]
  We propose blur, noise, and compression robust GAN (BNCR-GAN) to learn a clean image generator directly from degraded images. Inspired by NR-GAN, BNCR-GAN uses a multiple-generator model composed of image, blur- Kernel, noise, and quality-factor generators. We demonstrate the effectiveness of BNCR-GAN through large-scale comparative studies on CIFAR-10 and a generality analysis on FFHQ.
  arXiv  Detail & Related papers  (2020-03-17T17:56:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.