Perception- and Fidelity-aware Reduced-Reference Super-Resolution Image Quality Assessment

• URL: http://arxiv.org/abs/2405.09472v2
• Date: Sat, 27 Jul 2024 17:16:48 GMT
• Title: Perception- and Fidelity-aware Reduced-Reference Super-Resolution Image Quality Assessment
• Authors: Xinying Lin, Xuyang Liu, Hong Yang, Xiaohai He, Honggang Chen,
• Abstract summary: We propose a novel dual-branch reduced-reference SR-IQA network, ie, Perception- and Fidelity-aware SR-IQA (PFIQA) PFIQA outperforms current state-of-the-art models across three widely-used SR-IQA benchmarks.
• Score: 25.88845910499606
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the advent of image super-resolution (SR) algorithms, how to evaluate the quality of generated SR images has become an urgent task. Although full-reference methods perform well in SR image quality assessment (SR-IQA), their reliance on high-resolution (HR) images limits their practical applicability. Leveraging available reconstruction information as much as possible for SR-IQA, such as low-resolution (LR) images and the scale factors, is a promising way to enhance assessment performance for SR-IQA without HR for reference. In this letter, we attempt to evaluate the perceptual quality and reconstruction fidelity of SR images considering LR images and scale factors. Specifically, we propose a novel dual-branch reduced-reference SR-IQA network, \ie, Perception- and Fidelity-aware SR-IQA (PFIQA). The perception-aware branch evaluates the perceptual quality of SR images by leveraging the merits of global modeling of Vision Transformer (ViT) and local relation of ResNet, and incorporating the scale factor to enable comprehensive visual perception. Meanwhile, the fidelity-aware branch assesses the reconstruction fidelity between LR and SR images through their visual perception. The combination of the two branches substantially aligns with the human visual system, enabling a comprehensive SR image evaluation. Experimental results indicate that our PFIQA outperforms current state-of-the-art models across three widely-used SR-IQA benchmarks. Notably, PFIQA excels in assessing the quality of real-world SR images.

Related papers

• Study of Subjective and Objective Quality in Super-Resolution Enhanced Broadcast Images on a Novel SR-IQA Dataset [4.770359059226373]
  Super-Resolution (SR), a key consumer technology, is essential to display low-quality broadcast content on high-resolution screens in full-screen format. evaluating the quality of SR images generated from low-quality sources, such as SR-enhanced broadcast content, is challenging. We introduce a new IQA dataset for SR broadcast images in both 2K and 4K resolutions.
  arXiv  Detail & Related papers  (2024-09-26T01:07:15Z)
• ICF-SRSR: Invertible scale-Conditional Function for Self-Supervised Real-world Single Image Super-Resolution [60.90817228730133]
  Single image super-resolution (SISR) is a challenging problem that aims to up-sample a given low-resolution (LR) image to a high-resolution (HR) counterpart. Recent approaches are trained on simulated LR images degraded by simplified down-sampling operators. We propose a novel Invertible scale-Conditional Function (ICF) which can scale an input image and then restore the original input with different scale conditions.
  arXiv  Detail & Related papers  (2023-07-24T12:42:45Z)
• Scale Guided Hypernetwork for Blind Super-Resolution Image Quality Assessment [2.4366811507669124]
  Existing blind SR image quality assessment (IQA) metrics merely focus on visual characteristics of super-resolution images. We propose a scale guided hypernetwork framework that evaluates SR image quality in a scale-adaptive manner.
  arXiv  Detail & Related papers  (2023-06-04T16:17:19Z)
• CiaoSR: Continuous Implicit Attention-in-Attention Network for Arbitrary-Scale Image Super-Resolution [158.2282163651066]
  This paper proposes a continuous implicit attention-in-attention network, called CiaoSR. We explicitly design an implicit attention network to learn the ensemble weights for the nearby local features. We embed a scale-aware attention in this implicit attention network to exploit additional non-local information.
  arXiv  Detail & Related papers  (2022-12-08T15:57:46Z)
• Hierarchical Similarity Learning for Aliasing Suppression Image Super-Resolution [64.15915577164894]
  A hierarchical image super-resolution network (HSRNet) is proposed to suppress the influence of aliasing. HSRNet achieves better quantitative and visual performance than other works, and remits the aliasing more effectively.
  arXiv  Detail & Related papers  (2022-06-07T14:55:32Z)
• Textural-Structural Joint Learning for No-Reference Super-Resolution Image Quality Assessment [59.91741119995321]
  We develop a dual stream network to jointly explore the textural and structural information for quality prediction, dubbed TSNet. By mimicking the human vision system (HVS) that pays more attention to the significant areas of the image, we develop the spatial attention mechanism to make the visual-sensitive areas more distinguishable. Experimental results show the proposed TSNet predicts the visual quality more accurate than the state-of-the-art IQA methods, and demonstrates better consistency with the human's perspective.
  arXiv  Detail & Related papers  (2022-05-27T09:20:06Z)
• SPQE: Structure-and-Perception-Based Quality Evaluation for Image Super-Resolution [24.584839578742237]
  Super-Resolution technique has greatly improved the visual quality of images by enhancing their resolutions. It also calls for an efficient SR Image Quality Assessment (SR-IQA) to evaluate those algorithms or their generated images. In emerging deep-learning-based SR, a generated high-quality, visually pleasing image may have different structures from its corresponding low-quality image.
  arXiv  Detail & Related papers  (2022-05-07T07:52:55Z)
• Learning Conditional Knowledge Distillation for Degraded-Reference Image Quality Assessment [157.1292674649519]
  We propose a practical solution named degraded-reference IQA (DR-IQA) DR-IQA exploits the inputs of IR models, degraded images, as references. Our results can even be close to the performance of full-reference settings.
  arXiv  Detail & Related papers  (2021-08-18T02:35:08Z)
• Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling [139.25215100378284]
  We propose a hierarchical conditional flow (HCFlow) as a unified framework for image SR and image rescaling. HCFlow learns a mapping between HR and LR image pairs by modelling the distribution of the LR image and the rest high-frequency component simultaneously. To further enhance the performance, other losses such as perceptual loss and GAN loss are combined with the commonly used negative log-likelihood loss in training.
  arXiv  Detail & Related papers  (2021-08-11T16:11:01Z)
• Blind Quality Assessment for Image Superresolution Using Deep Two-Stream Convolutional Networks [41.558981828761574]
  We propose a no-reference/blind deep neural network-based SR image quality assessor (DeepSRQ) To learn more discriminative feature representations of various distorted SR images, the proposed DeepSRQ is a two-stream convolutional network. Experimental results on three publicly available SR image quality databases demonstrate the effectiveness and generalization ability of our proposed DeepSRQ.
  arXiv  Detail & Related papers  (2020-04-13T19:14:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.