Subjective Annotation for a Frame Interpolation Benchmark using Artefact Amplification

• URL: http://arxiv.org/abs/2001.06409v2
• Date: Tue, 28 Apr 2020 19:06:04 GMT
• Title: Subjective Annotation for a Frame Interpolation Benchmark using Artefact Amplification
• Authors: Hui Men, Vlad Hosu, Hanhe Lin, Andr\'es Bruhn and Dietmar Saupe
• Abstract summary: For image quality assessment, the actual quality experienced by the user cannot be fully deduced from simple measures. We conducted a subjective quality assessment crowdscouring study for the interpolated frames provided by one of the optical flow benchmarks. As a first step, we proposed such a new full-reference method, called WAE-IQA.
• Score: 6.544757635738911
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Current benchmarks for optical flow algorithms evaluate the estimation either directly by comparing the predicted flow fields with the ground truth or indirectly by using the predicted flow fields for frame interpolation and then comparing the interpolated frames with the actual frames. In the latter case, objective quality measures such as the mean squared error are typically employed. However, it is well known that for image quality assessment, the actual quality experienced by the user cannot be fully deduced from such simple measures. Hence, we conducted a subjective quality assessment crowdscouring study for the interpolated frames provided by one of the optical flow benchmarks, the Middlebury benchmark. We collected forced-choice paired comparisons between interpolated images and corresponding ground truth. To increase the sensitivity of observers when judging minute difference in paired comparisons we introduced a new method to the field of full-reference quality assessment, called artefact amplification. From the crowdsourcing data, we reconstructed absolute quality scale values according to Thurstone's model. As a result, we obtained a re-ranking of the 155 participating algorithms w.r.t. the visual quality of the interpolated frames. This re-ranking not only shows the necessity of visual quality assessment as another evaluation metric for optical flow and frame interpolation benchmarks, the results also provide the ground truth for designing novel image quality assessment (IQA) methods dedicated to perceptual quality of interpolated images. As a first step, we proposed such a new full-reference method, called WAE-IQA. By weighing the local differences between an interpolated image and its ground truth WAE-IQA performed slightly better than the currently best FR-IQA approach from the literature.

Related papers

• Foundation Models Boost Low-Level Perceptual Similarity Metrics [6.226609932118124]
  For full-reference image quality assessment (FR-IQA) using deep-learning approaches, the perceptual similarity score between a distorted image and a reference image is typically computed as a distance measure between features extracted from a pretrained CNN or more recently, a Transformer network. This work explores the potential of utilizing the intermediate features of these foundation models, which have largely been unexplored so far in the design of low-level perceptual similarity metrics.
  arXiv  Detail & Related papers  (2024-09-11T22:32:12Z)
• Attention Down-Sampling Transformer, Relative Ranking and Self-Consistency for Blind Image Quality Assessment [17.04649536069553]
  No-reference image quality assessment is a challenging domain that addresses estimating image quality without the original reference. We introduce an improved mechanism to extract local and non-local information from images via different transformer encoders and CNNs. A self-consistency approach to self-supervision is presented, explicitly addressing the degradation of no-reference image quality assessment (NR-IQA) models.
  arXiv  Detail & Related papers  (2024-09-11T09:08:43Z)
• Adaptive Image Quality Assessment via Teaching Large Multimodal Model to Compare [99.57567498494448]
  We introduce Compare2Score, an all-around LMM-based no-reference IQA model. During training, we generate scaled-up comparative instructions by comparing images from the same IQA dataset. Experiments on nine IQA datasets validate that the Compare2Score effectively bridges text-defined comparative levels during training.
  arXiv  Detail & Related papers  (2024-05-29T17:26:09Z)
• Beyond MOS: Subjective Image Quality Score Preprocessing Method Based on Perceptual Similarity [2.290956583394892]
  ITU-R BT.500, ITU-T P.910, and ITU-T P.913 have been standardized to clean up the original opinion scores. PSP exploit the perceptual similarity between images to alleviate subjective bias in less annotated scenarios.
  arXiv  Detail & Related papers  (2024-04-30T16:01:14Z)
• OCAI: Improving Optical Flow Estimation by Occlusion and Consistency Aware Interpolation [55.676358801492114]
  We propose OCAI, a method that supports robust frame ambiguities by generating intermediate video frames alongside optical flows in between. Our evaluations demonstrate superior quality and enhanced optical flow accuracy on established benchmarks such as Sintel and KITTI.
  arXiv  Detail & Related papers  (2024-03-26T20:23:48Z)
• Pairwise Comparisons Are All You Need [22.798716660911833]
  Blind image quality assessment (BIQA) approaches often fall short in real-world scenarios due to their reliance on a generic quality standard applied uniformly across diverse images. This paper introduces PICNIQ, a pairwise comparison framework designed to bypass the limitations of conventional BIQA. By employing psychometric scaling algorithms, PICNIQ transforms pairwise comparisons into just-objectionable-difference (JOD) quality scores, offering a granular and interpretable measure of image quality.
  arXiv  Detail & Related papers  (2024-03-13T23:43:36Z)
• Comparison of No-Reference Image Quality Models via MAP Estimation in Diffusion Latents [99.19391983670569]
  We show that NR-IQA models can be plugged into the maximum a posteriori (MAP) estimation framework for image enhancement. Different NR-IQA models are likely to induce different enhanced images, which are ultimately subject to psychophysical testing. This leads to a new computational method for comparing NR-IQA models within the analysis-by-synthesis framework.
  arXiv  Detail & Related papers  (2024-03-11T03:35:41Z)
• PIQI: Perceptual Image Quality Index based on Ensemble of Gaussian Process Regression [2.9412539021452715]
  Perceptual Image Quality Index (PIQI) is proposed to assess the quality of digital images. The performance of the PIQI is checked on six benchmark databases and compared with twelve state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-16T06:44:17Z)
• Learning Transformer Features for Image Quality Assessment [53.51379676690971]
  We propose a unified IQA framework that utilizes CNN backbone and transformer encoder to extract features. The proposed framework is compatible with both FR and NR modes and allows for a joint training scheme.
  arXiv  Detail & Related papers  (2021-12-01T13:23:00Z)
• Image Quality Assessment using Contrastive Learning [50.265638572116984]
  We train a deep Convolutional Neural Network (CNN) using a contrastive pairwise objective to solve the auxiliary problem. We show through extensive experiments that CONTRIQUE achieves competitive performance when compared to state-of-the-art NR image quality models. Our results suggest that powerful quality representations with perceptual relevance can be obtained without requiring large labeled subjective image quality datasets.
  arXiv  Detail & Related papers  (2021-10-25T21:01:00Z)
• TimeLens: Event-based Video Frame Interpolation [54.28139783383213]
  We introduce Time Lens, a novel indicates equal contribution method that leverages the advantages of both synthesis-based and flow-based approaches. We show an up to 5.21 dB improvement in terms of PSNR over state-of-the-art frame-based and event-based methods.
  arXiv  Detail & Related papers  (2021-06-14T10:33:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.