Enhancing Perceptual Loss with Adversarial Feature Matching for Super-Resolution

• URL: http://arxiv.org/abs/2005.07502v1
• Date: Fri, 15 May 2020 12:36:54 GMT
• Title: Enhancing Perceptual Loss with Adversarial Feature Matching for Super-Resolution
• Authors: Akella Ravi Tej, Shirsendu Sukanta Halder, Arunav Pratap Shandeelya, Vinod Pankajakshan
• Abstract summary: Single image super-resolution (SISR) is an ill-posed problem with an indeterminate number of valid solutions. We show that the root cause of these pattern artifacts can be traced back to a mismatch between the pre-training objective of perceptual loss and the super-resolved objective.
• Score: 5.258555266148511
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Single image super-resolution (SISR) is an ill-posed problem with an indeterminate number of valid solutions. Solving this problem with neural networks would require access to extensive experience, either presented as a large training set over natural images or a condensed representation from another pre-trained network. Perceptual loss functions, which belong to the latter category, have achieved breakthrough success in SISR and several other computer vision tasks. While perceptual loss plays a central role in the generation of photo-realistic images, it also produces undesired pattern artifacts in the super-resolved outputs. In this paper, we show that the root cause of these pattern artifacts can be traced back to a mismatch between the pre-training objective of perceptual loss and the super-resolution objective. To address this issue, we propose to augment the existing perceptual loss formulation with a novel content loss function that uses the latent features of a discriminator network to filter the unwanted artifacts across several levels of adversarial similarity. Further, our modification has a stabilizing effect on non-convex optimization in adversarial training. The proposed approach offers notable gains in perceptual quality based on an extensive human evaluation study and a competent reconstruction fidelity when tested on objective evaluation metrics.

Related papers

• Improving Neural Surface Reconstruction with Feature Priors from Multi-View Image [87.00660347447494]
  Recent advancements in Neural Surface Reconstruction (NSR) have significantly improved multi-view reconstruction when coupled with volume rendering. We propose an investigation into feature-level consistent loss, aiming to harness valuable feature priors from diverse pretext visual tasks. Our results, analyzed on DTU and EPFL, reveal that feature priors from image matching and multi-view stereo datasets outperform other pretext tasks.
  arXiv  Detail & Related papers  (2024-08-04T16:09:46Z)
• Hierarchical Similarity Learning for Aliasing Suppression Image Super-Resolution [64.15915577164894]
  A hierarchical image super-resolution network (HSRNet) is proposed to suppress the influence of aliasing. HSRNet achieves better quantitative and visual performance than other works, and remits the aliasing more effectively.
  arXiv  Detail & Related papers  (2022-06-07T14:55:32Z)
• Dual Perceptual Loss for Single Image Super-Resolution Using ESRGAN [13.335546116599494]
  This paper proposes a method called Dual Perceptual Loss (DP Loss) to replace the original perceptual loss to solve the problem of single image super-resolution reconstruction. Due to the complementary property between the VGG features and the ResNet features, the proposed DP Loss considers the advantages of learning two features simultaneously. The qualitative and quantitative analysis on benchmark datasets demonstrates the superiority of our proposed method over state-of-the-art super-resolution methods.
  arXiv  Detail & Related papers  (2022-01-17T12:42:56Z)
• A-ESRGAN: Training Real-World Blind Super-Resolution with Attention U-Net Discriminators [0.0]
  Blind image super-resolution(SR) is a long-standing task in CV that aims to restore low-resolution images suffering from unknown and complex distortions. We present A-ESRGAN, a GAN model for blind SR tasks featuring an attention U-Net based, multi-scale discriminator.
  arXiv  Detail & Related papers  (2021-12-19T02:50:23Z)
• Learning GAN-based Foveated Reconstruction to Recover Perceptually Important Image Features [0.0]
  We consider the problem of efficiently guiding the training of foveated reconstruction techniques. Our primary goal is to make the training procedure less sensitive to distortions that humans cannot detect. Our evaluations revealed significant improvements in the perceived image reconstruction quality compared with the standard GAN-based training approach.
  arXiv  Detail & Related papers  (2021-08-07T18:39:49Z)
• Training a Better Loss Function for Image Restoration [17.20936270604533]
  We show that a single natural image is sufficient to train a lightweight feature extractor that outperforms state-of-the-art loss functions in single image super resolution. We propose a novel Multi-Scale Discriminative Feature (MDF) loss comprising a series of discriminators, trained to penalize errors introduced by a generator.
  arXiv  Detail & Related papers  (2021-03-26T17:29:57Z)
• Invertible Image Rescaling [118.2653765756915]
  We develop an Invertible Rescaling Net (IRN) to produce visually-pleasing low-resolution images. We capture the distribution of the lost information using a latent variable following a specified distribution in the downscaling process.
  arXiv  Detail & Related papers  (2020-05-12T09:55:53Z)
• Real-world Person Re-Identification via Degradation Invariance Learning [111.86722193694462]
  Person re-identification (Re-ID) in real-world scenarios usually suffers from various degradation factors, e.g., low-resolution, weak illumination, blurring and adverse weather. We propose a degradation invariance learning framework for real-world person Re-ID. By introducing a self-supervised disentangled representation learning strategy, our method is able to simultaneously extract identity-related robust features.
  arXiv  Detail & Related papers  (2020-04-10T07:58:50Z)
• Gated Fusion Network for Degraded Image Super Resolution [78.67168802945069]
  We propose a dual-branch convolutional neural network to extract base features and recovered features separately. By decomposing the feature extraction step into two task-independent streams, the dual-branch model can facilitate the training process.
  arXiv  Detail & Related papers  (2020-03-02T13:28:32Z)
• When Relation Networks meet GANs: Relation GANs with Triplet Loss [110.7572918636599]
  Training stability is still a lingering concern of generative adversarial networks (GANs) In this paper, we explore a relation network architecture for the discriminator and design a triplet loss which performs better generalization and stability. Experiments on benchmark datasets show that the proposed relation discriminator and new loss can provide significant improvement on variable vision tasks.
  arXiv  Detail & Related papers  (2020-02-24T11:35:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.