Beyond Image Super-Resolution for Image Recognition with Task-Driven Perceptual Loss

• URL: http://arxiv.org/abs/2404.01692v2
• Date: Thu, 4 Apr 2024 08:07:22 GMT
• Title: Beyond Image Super-Resolution for Image Recognition with Task-Driven Perceptual Loss
• Authors: Jaeha Kim, Junghun Oh, Kyoung Mu Lee,
• Abstract summary: Super-Resolution for Image Recognition (SR4IR) guides the generation of SR images beneficial to image recognition performance. In this paper, we demonstrate that our SR4IR achieves outstanding task performance by generating SR images useful for a specific image recognition task.
• Score: 47.36902705025445
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In real-world scenarios, image recognition tasks, such as semantic segmentation and object detection, often pose greater challenges due to the lack of information available within low-resolution (LR) content. Image super-resolution (SR) is one of the promising solutions for addressing the challenges. However, due to the ill-posed property of SR, it is challenging for typical SR methods to restore task-relevant high-frequency contents, which may dilute the advantage of utilizing the SR method. Therefore, in this paper, we propose Super-Resolution for Image Recognition (SR4IR) that effectively guides the generation of SR images beneficial to achieving satisfactory image recognition performance when processing LR images. The critical component of our SR4IR is the task-driven perceptual (TDP) loss that enables the SR network to acquire task-specific knowledge from a network tailored for a specific task. Moreover, we propose a cross-quality patch mix and an alternate training framework that significantly enhances the efficacy of the TDP loss by addressing potential problems when employing the TDP loss. Through extensive experiments, we demonstrate that our SR4IR achieves outstanding task performance by generating SR images useful for a specific image recognition task, including semantic segmentation, object detection, and image classification. The implementation code is available at https://github.com/JaehaKim97/SR4IR.

Related papers

• Rethinking Image Super-Resolution from Training Data Perspectives [54.28824316574355]
  We investigate the understudied effect of the training data used for image super-resolution (SR) With this, we propose an automated image evaluation pipeline. We find that datasets with (i) low compression artifacts, (ii) high within-image diversity as judged by the number of different objects, and (iii) a large number of images from ImageNet or PASS all positively affect SR performance.
  arXiv  Detail & Related papers  (2024-09-01T16:25:04Z)
• UnmixingSR: Material-aware Network with Unsupervised Unmixing as Auxiliary Task for Hyperspectral Image Super-resolution [5.167168688234238]
  This paper proposes a component-aware hyperspectral image (HIS) super-resolution network called UnmixingSR. We use the bond between LR abundances and HR abundances to boost the stability of our method in solving SR problems. Experimental results show that unmixing process as an auxiliary task incorporated into the SR problem is feasible and rational.
  arXiv  Detail & Related papers  (2024-07-09T03:41:02Z)
• SeD: Semantic-Aware Discriminator for Image Super-Resolution [20.646975821512395]
  Generative Adversarial Networks (GANs) have been widely used to recover vivid textures in image super-resolution (SR) tasks. One discriminator is utilized to enable the SR network to learn the distribution of real-world high-quality images in an adversarial training manner. We propose the simple and effective Semantic-aware Discriminator ( SeD) SeD encourages the SR network to learn the fine-grained distributions by introducing the semantics of images as a condition.
  arXiv  Detail & Related papers  (2024-02-29T17:38:54Z)
• ICF-SRSR: Invertible scale-Conditional Function for Self-Supervised Real-world Single Image Super-Resolution [60.90817228730133]
  Single image super-resolution (SISR) is a challenging problem that aims to up-sample a given low-resolution (LR) image to a high-resolution (HR) counterpart. Recent approaches are trained on simulated LR images degraded by simplified down-sampling operators. We propose a novel Invertible scale-Conditional Function (ICF) which can scale an input image and then restore the original input with different scale conditions.
  arXiv  Detail & Related papers  (2023-07-24T12:42:45Z)
• CiaoSR: Continuous Implicit Attention-in-Attention Network for Arbitrary-Scale Image Super-Resolution [158.2282163651066]
  This paper proposes a continuous implicit attention-in-attention network, called CiaoSR. We explicitly design an implicit attention network to learn the ensemble weights for the nearby local features. We embed a scale-aware attention in this implicit attention network to exploit additional non-local information.
  arXiv  Detail & Related papers  (2022-12-08T15:57:46Z)
• SRTGAN: Triplet Loss based Generative Adversarial Network for Real-World Super-Resolution [13.897062992922029]
  An alternative solution called Single Image Super-Resolution (SISR) is a software-driven approach that aims to take a Low-Resolution (LR) image and obtain the HR image. We introduce a new triplet-based adversarial loss function that exploits the information provided in the LR image by using it as a negative sample. We propose to fuse the adversarial loss, content loss, perceptual loss, and quality loss to obtain Super-Resolution (SR) image with high perceptual fidelity.
  arXiv  Detail & Related papers  (2022-11-22T11:17:07Z)
• Hierarchical Similarity Learning for Aliasing Suppression Image Super-Resolution [64.15915577164894]
  A hierarchical image super-resolution network (HSRNet) is proposed to suppress the influence of aliasing. HSRNet achieves better quantitative and visual performance than other works, and remits the aliasing more effectively.
  arXiv  Detail & Related papers  (2022-06-07T14:55:32Z)
• FAN: Frequency Aggregation Network for Real Image Super-resolution [33.30542701042704]
  Single image super-resolution (SISR) aims to recover the high-resolution (HR) image from its low-resolution (LR) input image. We propose FAN, a frequency aggregation network, to address the real-world image super-resolu-tion problem.
  arXiv  Detail & Related papers  (2020-09-30T10:18:41Z)
• DDet: Dual-path Dynamic Enhancement Network for Real-World Image Super-Resolution [69.2432352477966]
  Real image super-resolution(Real-SR) focus on the relationship between real-world high-resolution(HR) and low-resolution(LR) image. In this article, we propose a Dual-path Dynamic Enhancement Network(DDet) for Real-SR. Unlike conventional methods which stack up massive convolutional blocks for feature representation, we introduce a content-aware framework to study non-inherently aligned image pair.
  arXiv  Detail & Related papers  (2020-02-25T18:24:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.