A New Super-Resolution Measurement of Perceptual Quality and Fidelity

• URL: http://arxiv.org/abs/2303.06207v1
• Date: Fri, 10 Mar 2023 21:08:24 GMT
• Title: A New Super-Resolution Measurement of Perceptual Quality and Fidelity
• Authors: Sheng Cheng
• Abstract summary: Super-resolution results are usually measured by full-reference image quality metrics or human rating scores. In this work, we analyze the evaluation problem based on the one-to-many mapping nature of super-resolution. We show that the proposed metric is highly correlated with the human perceptual quality, and better than most existing metrics.
• Score: 2.901173495131855
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Super-resolution results are usually measured by full-reference image quality metrics or human rating scores. However, these evaluation methods are general image quality measurement, and do not account for the nature of the super-resolution problem. In this work, we analyze the evaluation problem based on the one-to-many mapping nature of super-resolution, and propose a novel distribution-based metric for super-resolution. Starting from the distribution distance, we derive the proposed metric to make it accessible and easy to compute. Through a human subject study on super-resolution, we show that the proposed metric is highly correlated with the human perceptual quality, and better than most existing metrics. Moreover, the proposed metric has a higher correlation with the fidelity measure compared to the perception-based metrics. To understand the properties of the proposed metric, we conduct extensive evaluation in terms of its design choices, and show that the metric is robust to its design choices. Finally, we show that the metric can be used to train super-resolution networks for better perceptual quality.

Related papers

• Towards an Improved Metric for Evaluating Disentangled Representations [0.6946415403594184]
  Disentangled representation learning plays a pivotal role in making representations controllable, interpretable and transferable. Despite its significance in the domain, the quest for reliable and consistent quantitative disentanglement metric remains a major challenge. We propose a new framework for quantifying disentanglement, introducing a metric entitled emphEDI, that leverages the intuitive concept of emphexclusivity and improved factor-code relationship.
  arXiv  Detail & Related papers  (2024-10-04T00:32:59Z)
• Is Reference Necessary in the Evaluation of NLG Systems? When and Where? [58.52957222172377]
  We show that reference-free metrics exhibit a higher correlation with human judgment and greater sensitivity to deficiencies in language quality. Our study can provide insight into the appropriate application of automatic metrics and the impact of metric choice on evaluation performance.
  arXiv  Detail & Related papers  (2024-03-21T10:31:11Z)
• What is the Best Automated Metric for Text to Motion Generation? [19.71712698183703]
  There is growing interest in generating skeleton-based human motions from natural language descriptions. Human evaluation is the ultimate accuracy measure for this task, and automated metrics should correlate well with human quality judgments. This paper systematically studies which metrics best align with human evaluations and proposes new metrics that align even better.
  arXiv  Detail & Related papers  (2023-09-19T01:59:54Z)
• Metrics reloaded: Recommendations for image analysis validation [59.60445111432934]
  Metrics Reloaded is a comprehensive framework guiding researchers in the problem-aware selection of metrics. The framework was developed in a multi-stage Delphi process and is based on the novel concept of a problem fingerprint. Based on the problem fingerprint, users are guided through the process of choosing and applying appropriate validation metrics.
  arXiv  Detail & Related papers  (2022-06-03T15:56:51Z)
• A study of deep perceptual metrics for image quality assessment [3.254879465902239]
  We study perceptual metrics based on deep neural networks for tackling the Image Quality Assessment (IQA) task. We propose our multi-resolution perceptual metric (MR-Perceptual) that allows us to aggregate perceptual information at different resolutions.
  arXiv  Detail & Related papers  (2022-02-17T14:52:53Z)
• QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarization [116.56171113972944]
  We show that carefully choosing the components of a QA-based metric is critical to performance. Our solution improves upon the best-performing entailment-based metric and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-12-16T00:38:35Z)
• Full-resolution quality assessment for pansharpening [0.0]
  A reliable quality assessment procedure for pansharpening methods is of critical importance for the development of the related solutions. We introduce a protocol, namely the reprojection protocol, which allows to handle the spectral fidelity problem. On the other side, a new index of the spatial consistency between the pansharpened image and the panchromatic band at full resolution is proposed.
  arXiv  Detail & Related papers  (2021-08-13T09:35:45Z)
• GO FIGURE: A Meta Evaluation of Factuality in Summarization [131.1087461486504]
  We introduce GO FIGURE, a meta-evaluation framework for evaluating factuality evaluation metrics. Our benchmark analysis on ten factuality metrics reveals that our framework provides a robust and efficient evaluation. It also reveals that while QA metrics generally improve over standard metrics that measure factuality across domains, performance is highly dependent on the way in which questions are generated.
  arXiv  Detail & Related papers  (2020-10-24T08:30:20Z)
• Towards Question-Answering as an Automatic Metric for Evaluating the Content Quality of a Summary [65.37544133256499]
  We propose a metric to evaluate the content quality of a summary using question-answering (QA) We demonstrate the experimental benefits of QA-based metrics through an analysis of our proposed metric, QAEval.
  arXiv  Detail & Related papers  (2020-10-01T15:33:09Z)
• Full Reference Screen Content Image Quality Assessment by Fusing Multi-level Structure Similarity [27.971146869941112]
  This paper advocates a novel solution to measure structure similarity "globally" from the perspective of sparse representation. To perform multi-level quality assessment in accordance with the real HVS, the above-mentioned global metric will be integrated with the conventional local ones by resorting to the newly devised selective deep fusion network.
  arXiv  Detail & Related papers  (2020-08-07T10:20:25Z)
• Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metrics [64.88815792555451]
  We show that current methods for judging metrics are highly sensitive to the translations used for assessment. We develop a method for thresholding performance improvement under an automatic metric against human judgements.
  arXiv  Detail & Related papers  (2020-06-11T09:12:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.