DeSRA: Detect and Delete the Artifacts of GAN-based Real-World Super-Resolution Models

• URL: http://arxiv.org/abs/2307.02457v1
• Date: Wed, 5 Jul 2023 17:31:44 GMT
• Title: DeSRA: Detect and Delete the Artifacts of GAN-based Real-World Super-Resolution Models
• Authors: Liangbin Xie, Xintao Wang, Xiangyu Chen, Gen Li, Ying Shan, Jiantao Zhou, Chao Dong
• Abstract summary: Image super-resolution (SR) with generative adversarial networks (GAN) has achieved great success in restoring realistic details. It is notorious that GAN-based SR models will inevitably produce unpleasant and undesirable artifacts. In this paper, we analyze the cause and characteristics of the GAN artifacts produced in unseen test data without ground-truths. We develop a novel method, namely, DeSRA, to Detect and then Delete those SR Artifacts in practice.
• Score: 41.60982753592467
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image super-resolution (SR) with generative adversarial networks (GAN) has achieved great success in restoring realistic details. However, it is notorious that GAN-based SR models will inevitably produce unpleasant and undesirable artifacts, especially in practical scenarios. Previous works typically suppress artifacts with an extra loss penalty in the training phase. They only work for in-distribution artifact types generated during training. When applied in real-world scenarios, we observe that those improved methods still generate obviously annoying artifacts during inference. In this paper, we analyze the cause and characteristics of the GAN artifacts produced in unseen test data without ground-truths. We then develop a novel method, namely, DeSRA, to Detect and then Delete those SR Artifacts in practice. Specifically, we propose to measure a relative local variance distance from MSE-SR results and GAN-SR results, and locate the problematic areas based on the above distance and semantic-aware thresholds. After detecting the artifact regions, we develop a finetune procedure to improve GAN-based SR models with a few samples, so that they can deal with similar types of artifacts in more unseen real data. Equipped with our DeSRA, we can successfully eliminate artifacts from inference and improve the ability of SR models to be applied in real-world scenarios. The code will be available at https://github.com/TencentARC/DeSRA.

Related papers

• Enhanced Super-Resolution Training via Mimicked Alignment for Real-World Scenes [51.92255321684027]
  We propose a novel plug-and-play module designed to mitigate misalignment issues by aligning LR inputs with HR images during training. Specifically, our approach involves mimicking a novel LR sample that aligns with HR while preserving the characteristics of the original LR samples. We comprehensively evaluate our method on synthetic and real-world datasets, demonstrating its effectiveness across a spectrum of SR models.
  arXiv  Detail & Related papers  (2024-10-07T18:18:54Z)
• Context Enhancement with Reconstruction as Sequence for Unified Unsupervised Anomaly Detection [68.74469657656822]
  Unsupervised anomaly detection (AD) aims to train robust detection models using only normal samples. Recent research focuses on a unified unsupervised AD setting in which only one model is trained for all classes. We introduce a novel Reconstruction as Sequence (RAS) method, which enhances the contextual correspondence during feature reconstruction.
  arXiv  Detail & Related papers  (2024-09-10T07:37:58Z)
• Generalized Deepfake Attribution [1.4999444543328293]
  Generative Adversarial Networks (GANs) can produce countless variations of GAN models depending on the seed used. Existing methods for attributing deepfakes work well only if they have seen the specific GAN model during training. We propose a generalized deepfake attribution network (GDA-N et) to attribute fake images to their respective GAN architectures.
  arXiv  Detail & Related papers  (2024-06-26T12:04:09Z)
• Self-Adaptive Reality-Guided Diffusion for Artifact-Free Super-Resolution [47.29558685384506]
  Artifact-free super-resolution (SR) aims to translate low-resolution images into their high-resolution counterparts with a strict integrity of the original content. Traditional diffusion-based SR techniques are prone to artifact introduction during iterative procedures. We propose Self-Adaptive Reality-Guided Diffusion to identify and mitigate the propagation of artifacts.
  arXiv  Detail & Related papers  (2024-03-25T11:29:19Z)
• A General Method to Incorporate Spatial Information into Loss Functions for GAN-based Super-resolution Models [25.69505971220203]
  Generative Adversarial Networks (GANs) have shown great performance on super-resolution problems. GANs often introduce side effects into the outputs, such as unexpected artifacts and noises. We propose a general method that can be effectively used in most GAN-based super-resolution (SR) models by introducing essential spatial information into the training process.
  arXiv  Detail & Related papers  (2024-03-15T17:29:16Z)
• Data-Independent Operator: A Training-Free Artifact Representation Extractor for Generalizable Deepfake Detection [105.9932053078449]
  In this work, we show that, on the contrary, the small and training-free filter is sufficient to capture more general artifact representations. Due to its unbias towards both the training and test sources, we define it as Data-Independent Operator (DIO) to achieve appealing improvements on unseen sources. Our detector achieves a remarkable improvement of $13.3%$, establishing a new state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-03-11T15:22:28Z)
• SMaRt: Improving GANs with Score Matching Regularity [94.81046452865583]
  Generative adversarial networks (GANs) usually struggle in learning from highly diverse data, whose underlying manifold is complex. We show that score matching serves as a promising solution to this issue thanks to its capability of persistently pushing the generated data points towards the real data manifold. We propose to improve the optimization of GANs with score matching regularity (SMaRt)
  arXiv  Detail & Related papers  (2023-11-30T03:05:14Z)
• Details or Artifacts: A Locally Discriminative Learning Approach to Realistic Image Super-Resolution [28.00231586840797]
  Single image super-resolution (SISR) with generative adversarial networks (GAN) has recently attracted increasing attention due to its potentials to generate rich details. In this paper, we demonstrate that it is possible to train a GAN-based SISR model which can stably generate perceptually realistic details while inhibiting visual artifacts.
  arXiv  Detail & Related papers  (2022-03-17T09:35:50Z)
• Generalized Real-World Super-Resolution through Adversarial Robustness [107.02188934602802]
  We present Robust Super-Resolution, a method that leverages the generalization capability of adversarial attacks to tackle real-world SR. Our novel framework poses a paradigm shift in the development of real-world SR methods. By using a single robust model, we outperform state-of-the-art specialized methods on real-world benchmarks.
  arXiv  Detail & Related papers  (2021-08-25T22:43:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.