Interpretable Image Quality Assessment via CLIP with Multiple Antonym-Prompt Pairs

• URL: http://arxiv.org/abs/2308.13094v1
• Date: Thu, 24 Aug 2023 21:37:00 GMT
• Title: Interpretable Image Quality Assessment via CLIP with Multiple Antonym-Prompt Pairs
• Authors: Takamichi Miyata
• Abstract summary: No reference image quality assessment (NR-IQA) is a task to estimate the perceptual quality of an image without its corresponding original image. We propose a new zero-shot and interpretable NRIQA method that exploits the ability of a pre-trained vision model. Experimental results show that the proposed method outperforms existing zero-shot NR-IQA methods in terms of accuracy.
• Score: 1.6317061277457001
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: No reference image quality assessment (NR-IQA) is a task to estimate the perceptual quality of an image without its corresponding original image. It is even more difficult to perform this task in a zero-shot manner, i.e., without task-specific training. In this paper, we propose a new zero-shot and interpretable NRIQA method that exploits the ability of a pre-trained visionlanguage model to estimate the correlation between an image and a textual prompt. The proposed method employs a prompt pairing strategy and multiple antonym-prompt pairs corresponding to carefully selected descriptive features corresponding to the perceptual image quality. Thus, the proposed method is able to identify not only the perceptual quality evaluation of the image, but also the cause on which the quality evaluation is based. Experimental results show that the proposed method outperforms existing zero-shot NR-IQA methods in terms of accuracy and can evaluate the causes of perceptual quality degradation.

Related papers

• AI-Generated Image Quality Assessment Based on Task-Specific Prompt and Multi-Granularity Similarity [62.00987205438436]
  We propose a novel quality assessment method for AIGIs named TSP-MGS. It designs task-specific prompts and measures multi-granularity similarity between AIGIs and the prompts. Experiments on the commonly used AGIQA-1K and AGIQA-3K benchmarks demonstrate the superiority of the proposed TSP-MGS.
  arXiv  Detail & Related papers  (2024-11-25T04:47:53Z)
• Multi-Modal Prompt Learning on Blind Image Quality Assessment [65.0676908930946]
  Image Quality Assessment (IQA) models benefit significantly from semantic information, which allows them to treat different types of objects distinctly. Traditional methods, hindered by a lack of sufficiently annotated data, have employed the CLIP image-text pretraining model as their backbone to gain semantic awareness. Recent approaches have attempted to address this mismatch using prompt technology, but these solutions have shortcomings. This paper introduces an innovative multi-modal prompt-based methodology for IQA.
  arXiv  Detail & Related papers  (2024-04-23T11:45:32Z)
• QGFace: Quality-Guided Joint Training For Mixed-Quality Face Recognition [2.8519768339207356]
  We propose a novel quality-guided joint training approach for mixed-quality face recognition. Based on quality partition, classification-based method is employed for HQ data learning. For the LQ images which lack identity information, we learn them with self-supervised image-image contrastive learning.
  arXiv  Detail & Related papers  (2023-12-29T06:56:22Z)
• Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective [93.56647950778357]
  Blind image quality assessment (BIQA) predicts the human perception of image quality without any reference information. We develop a general and automated multitask learning scheme for BIQA to exploit auxiliary knowledge from other tasks.
  arXiv  Detail & Related papers  (2023-03-27T07:58:09Z)
• Conformer and Blind Noisy Students for Improved Image Quality Assessment [80.57006406834466]
  Learning-based approaches for perceptual image quality assessment (IQA) usually require both the distorted and reference image for measuring the perceptual quality accurately. In this work, we explore the performance of transformer-based full-reference IQA models. We also propose a method for IQA based on semi-supervised knowledge distillation from full-reference teacher models into blind student models.
  arXiv  Detail & Related papers  (2022-04-27T10:21:08Z)
• Image Quality Assessment using Contrastive Learning [50.265638572116984]
  We train a deep Convolutional Neural Network (CNN) using a contrastive pairwise objective to solve the auxiliary problem. We show through extensive experiments that CONTRIQUE achieves competitive performance when compared to state-of-the-art NR image quality models. Our results suggest that powerful quality representations with perceptual relevance can be obtained without requiring large labeled subjective image quality datasets.
  arXiv  Detail & Related papers  (2021-10-25T21:01:00Z)
• A survey on IQA [0.0]
  This article will review the concepts and metrics of image quality assessment and also video quality assessment. It briefly introduce some methods of full-reference and semi-reference image quality assessment, and focus on the non-reference image quality assessment methods based on deep learning.
  arXiv  Detail & Related papers  (2021-08-29T10:52:27Z)
• Learning Conditional Knowledge Distillation for Degraded-Reference Image Quality Assessment [157.1292674649519]
  We propose a practical solution named degraded-reference IQA (DR-IQA) DR-IQA exploits the inputs of IR models, degraded images, as references. Our results can even be close to the performance of full-reference settings.
  arXiv  Detail & Related papers  (2021-08-18T02:35:08Z)
• No-Reference Image Quality Assessment by Hallucinating Pristine Features [24.35220427707458]
  We propose a no-reference (NR) image quality assessment (IQA) method via feature level pseudo-reference (PR) hallucination. The effectiveness of our proposed method is demonstrated on four popular IQA databases.
  arXiv  Detail & Related papers  (2021-08-09T16:48:34Z)
• No-Reference Image Quality Assessment via Feature Fusion and Multi-Task Learning [29.19484863898778]
  Blind or no-reference image quality assessment (NR-IQA) is a fundamental, unsolved, and yet challenging problem. We propose a simple and yet effective general-purpose no-reference (NR) image quality assessment framework based on multi-task learning. Our model employs distortion types as well as subjective human scores to predict image quality.
  arXiv  Detail & Related papers  (2020-06-06T05:04:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.