DifAugGAN: A Practical Diffusion-style Data Augmentation for GAN-based Single Image Super-resolution

• URL: http://arxiv.org/abs/2311.18508v1
• Date: Thu, 30 Nov 2023 12:37:53 GMT
• Title: DifAugGAN: A Practical Diffusion-style Data Augmentation for GAN-based Single Image Super-resolution
• Authors: Axi Niu, Kang Zhang, Joshua Tian Jin Tee, Trung X. Pham, Jinqiu Sun, Chang D. Yoo, In So Kweon, Yanning Zhang
• Abstract summary: We propose a diffusion-style data augmentation scheme for GAN-based image super-resolution (SR) methods, known as DifAugGAN. It involves adapting the diffusion process in generative diffusion models for improving the calibration of the discriminator during training. Our DifAugGAN can be a Plug-and-Play strategy for current GAN-based SISR methods to improve the calibration of the discriminator and thus improve SR performance.
• Score: 88.13972071356422
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: It is well known the adversarial optimization of GAN-based image super-resolution (SR) methods makes the preceding SR model generate unpleasant and undesirable artifacts, leading to large distortion. We attribute the cause of such distortions to the poor calibration of the discriminator, which hampers its ability to provide meaningful feedback to the generator for learning high-quality images. To address this problem, we propose a simple but non-travel diffusion-style data augmentation scheme for current GAN-based SR methods, known as DifAugGAN. It involves adapting the diffusion process in generative diffusion models for improving the calibration of the discriminator during training motivated by the successes of data augmentation schemes in the field to achieve good calibration. Our DifAugGAN can be a Plug-and-Play strategy for current GAN-based SISR methods to improve the calibration of the discriminator and thus improve SR performance. Extensive experimental evaluations demonstrate the superiority of DifAugGAN over state-of-the-art GAN-based SISR methods across both synthetic and real-world datasets, showcasing notable advancements in both qualitative and quantitative results.

Related papers

• Degradation-Guided One-Step Image Super-Resolution with Diffusion Priors [75.24313405671433]
  Diffusion-based image super-resolution (SR) methods have achieved remarkable success by leveraging large pre-trained text-to-image diffusion models as priors. We introduce a novel one-step SR model, which significantly addresses the efficiency issue of diffusion-based SR methods. Unlike existing fine-tuning strategies, we designed a degradation-guided Low-Rank Adaptation (LoRA) module specifically for SR.
  arXiv  Detail & Related papers  (2024-09-25T16:15:21Z)
• Few-Shot Image Generation by Conditional Relaxing Diffusion Inversion [37.18537753482751]
  Conditional Diffusion Relaxing Inversion (CRDI) is designed to enhance distribution diversity in synthetic image generation. CRDI does not rely on fine-tuning based on only a few samples. It focuses on reconstructing each target image instance and expanding diversity through few-shot learning.
  arXiv  Detail & Related papers  (2024-07-09T21:58:26Z)
• Fortifying Fully Convolutional Generative Adversarial Networks for Image Super-Resolution Using Divergence Measures [17.517010701323823]
  Super-Resolution (SR) is a time-hallowed image processing problem. We introduce a fully-conversaal Generative Adrial Network (GAN)-based architecture for SR. We show that distinct convolutional features obtained at increasing depths of a GAN generator can be optimally combined by a set of learnable convex weights.
  arXiv  Detail & Related papers  (2024-04-09T13:19:43Z)
• ACDMSR: Accelerated Conditional Diffusion Models for Single Image Super-Resolution [84.73658185158222]
  We propose a diffusion model-based super-resolution method called ACDMSR. Our method adapts the standard diffusion model to perform super-resolution through a deterministic iterative denoising process. Our approach generates more visually realistic counterparts for low-resolution images, emphasizing its effectiveness in practical scenarios.
  arXiv  Detail & Related papers  (2023-07-03T06:49:04Z)
• Solving Diffusion ODEs with Optimal Boundary Conditions for Better Image Super-Resolution [82.50210340928173]
  randomness of diffusion models results in ineffectiveness and instability, making it challenging for users to guarantee the quality of SR results. We propose a plug-and-play sampling method that owns the potential to benefit a series of diffusion-based SR methods. The quality of SR results sampled by the proposed method with fewer steps outperforms the quality of results sampled by current methods with randomness from the same pre-trained diffusion-based SR model.
  arXiv  Detail & Related papers  (2023-05-24T17:09:54Z)
• DuDGAN: Improving Class-Conditional GANs via Dual-Diffusion [2.458437232470188]
  Class-conditional image generation using generative adversarial networks (GANs) has been investigated through various techniques. We propose a novel approach for class-conditional image generation using GANs called DuDGAN, which incorporates a dual diffusion-based noise injection process. Our method outperforms state-of-the-art conditional GAN models for image generation in terms of performance.
  arXiv  Detail & Related papers  (2023-05-24T07:59:44Z)
• Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data [125.7135706352493]
  Generative adversarial networks (GANs) typically require ample data for training in order to synthesize high-fidelity images. Recent studies have shown that training GANs with limited data remains formidable due to discriminator overfitting. This paper introduces a novel strategy called Adaptive Pseudo Augmentation (APA) to encourage healthy competition between the generator and the discriminator.
  arXiv  Detail & Related papers  (2021-11-12T18:13:45Z)
• Unsupervised Alternating Optimization for Blind Hyperspectral Imagery Super-resolution [40.350308926790255]
  This paper proposes an unsupervised blind HSI SR method to handle blind HSI fusion problem. We first propose an alternating optimization based deep framework to estimate the degeneration models and reconstruct the latent image. Then, a meta-learning based mechanism is further proposed to pre-train the network, which can effectively improve the speed and generalization ability.
  arXiv  Detail & Related papers  (2020-12-03T07:52:32Z)
• Characteristic Regularisation for Super-Resolving Face Images [81.84939112201377]
  Existing facial image super-resolution (SR) methods focus mostly on improving artificially down-sampled low-resolution (LR) imagery. Previous unsupervised domain adaptation (UDA) methods address this issue by training a model using unpaired genuine LR and HR data. This renders the model overstretched with two tasks: consistifying the visual characteristics and enhancing the image resolution. We formulate a method that joins the advantages of conventional SR and UDA models.
  arXiv  Detail & Related papers  (2019-12-30T16:27:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.