DeSRA: Detect and Delete the Artifacts of GAN-based Real-World
Super-Resolution Models
- URL: http://arxiv.org/abs/2307.02457v1
- Date: Wed, 5 Jul 2023 17:31:44 GMT
- Title: DeSRA: Detect and Delete the Artifacts of GAN-based Real-World
Super-Resolution Models
- Authors: Liangbin Xie, Xintao Wang, Xiangyu Chen, Gen Li, Ying Shan, Jiantao
Zhou, Chao Dong
- Abstract summary: Image super-resolution (SR) with generative adversarial networks (GAN) has achieved great success in restoring realistic details.
It is notorious that GAN-based SR models will inevitably produce unpleasant and undesirable artifacts.
In this paper, we analyze the cause and characteristics of the GAN artifacts produced in unseen test data without ground-truths.
We develop a novel method, namely, DeSRA, to Detect and then Delete those SR Artifacts in practice.
- Score: 41.60982753592467
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Image super-resolution (SR) with generative adversarial networks (GAN) has
achieved great success in restoring realistic details. However, it is notorious
that GAN-based SR models will inevitably produce unpleasant and undesirable
artifacts, especially in practical scenarios. Previous works typically suppress
artifacts with an extra loss penalty in the training phase. They only work for
in-distribution artifact types generated during training. When applied in
real-world scenarios, we observe that those improved methods still generate
obviously annoying artifacts during inference. In this paper, we analyze the
cause and characteristics of the GAN artifacts produced in unseen test data
without ground-truths. We then develop a novel method, namely, DeSRA, to Detect
and then Delete those SR Artifacts in practice. Specifically, we propose to
measure a relative local variance distance from MSE-SR results and GAN-SR
results, and locate the problematic areas based on the above distance and
semantic-aware thresholds. After detecting the artifact regions, we develop a
finetune procedure to improve GAN-based SR models with a few samples, so that
they can deal with similar types of artifacts in more unseen real data.
Equipped with our DeSRA, we can successfully eliminate artifacts from inference
and improve the ability of SR models to be applied in real-world scenarios. The
code will be available at https://github.com/TencentARC/DeSRA.
Related papers
- Generalized Deepfake Attribution [1.4999444543328293]
Generative Adversarial Networks (GANs) can produce countless variations of GAN models depending on the seed used.
Existing methods for attributing deepfakes work well only if they have seen the specific GAN model during training.
We propose a generalized deepfake attribution network (GDA-N et) to attribute fake images to their respective GAN architectures.
arXiv Detail & Related papers (2024-06-26T12:04:09Z) - Towards Realistic Data Generation for Real-World Super-Resolution [79.24617577528593]
RealDGen is an unsupervised learning data generation framework designed for real-world super-resolution.
We develop content and degradation extraction strategies, which are integrated into a novel content-degradation decoupled diffusion model.
Experiments demonstrate that RealDGen excels in generating large-scale, high-quality paired data that mirrors real-world degradations.
arXiv Detail & Related papers (2024-06-11T13:34:57Z) - Self-Adaptive Reality-Guided Diffusion for Artifact-Free Super-Resolution [47.29558685384506]
Artifact-free super-resolution (SR) aims to translate low-resolution images into their high-resolution counterparts with a strict integrity of the original content.
Traditional diffusion-based SR techniques are prone to artifact introduction during iterative procedures.
We propose Self-Adaptive Reality-Guided Diffusion to identify and mitigate the propagation of artifacts.
arXiv Detail & Related papers (2024-03-25T11:29:19Z) - A General Method to Incorporate Spatial Information into Loss Functions for GAN-based Super-resolution Models [25.69505971220203]
Generative Adversarial Networks (GANs) have shown great performance on super-resolution problems.
GANs often introduce side effects into the outputs, such as unexpected artifacts and noises.
We propose a general method that can be effectively used in most GAN-based super-resolution (SR) models by introducing essential spatial information into the training process.
arXiv Detail & Related papers (2024-03-15T17:29:16Z) - Data-Independent Operator: A Training-Free Artifact Representation
Extractor for Generalizable Deepfake Detection [105.9932053078449]
In this work, we show that, on the contrary, the small and training-free filter is sufficient to capture more general artifact representations.
Due to its unbias towards both the training and test sources, we define it as Data-Independent Operator (DIO) to achieve appealing improvements on unseen sources.
Our detector achieves a remarkable improvement of $13.3%$, establishing a new state-of-the-art performance.
arXiv Detail & Related papers (2024-03-11T15:22:28Z) - SMaRt: Improving GANs with Score Matching Regularity [94.81046452865583]
Generative adversarial networks (GANs) usually struggle in learning from highly diverse data, whose underlying manifold is complex.
We show that score matching serves as a promising solution to this issue thanks to its capability of persistently pushing the generated data points towards the real data manifold.
We propose to improve the optimization of GANs with score matching regularity (SMaRt)
arXiv Detail & Related papers (2023-11-30T03:05:14Z) - Details or Artifacts: A Locally Discriminative Learning Approach to
Realistic Image Super-Resolution [28.00231586840797]
Single image super-resolution (SISR) with generative adversarial networks (GAN) has recently attracted increasing attention due to its potentials to generate rich details.
In this paper, we demonstrate that it is possible to train a GAN-based SISR model which can stably generate perceptually realistic details while inhibiting visual artifacts.
arXiv Detail & Related papers (2022-03-17T09:35:50Z) - Generalized Real-World Super-Resolution through Adversarial Robustness [107.02188934602802]
We present Robust Super-Resolution, a method that leverages the generalization capability of adversarial attacks to tackle real-world SR.
Our novel framework poses a paradigm shift in the development of real-world SR methods.
By using a single robust model, we outperform state-of-the-art specialized methods on real-world benchmarks.
arXiv Detail & Related papers (2021-08-25T22:43:20Z) - Bridging the Gap Between Clean Data Training and Real-World Inference
for Spoken Language Understanding [76.89426311082927]
Existing models are trained on clean data, which causes a textitgap between clean data training and real-world inference.
We propose a method from the perspective of domain adaptation, by which both high- and low-quality samples are embedding into similar vector space.
Experiments on the widely-used dataset, Snips, and large scale in-house dataset (10 million training examples) demonstrate that this method not only outperforms the baseline models on real-world (noisy) corpus but also enhances the robustness, that is, it produces high-quality results under a noisy environment.
arXiv Detail & Related papers (2021-04-13T17:54:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.