Bridging the Gap Between Saliency Prediction and Image Quality Assessment

• URL: http://arxiv.org/abs/2405.04997v1
• Date: Wed, 8 May 2024 12:04:43 GMT
• Title: Bridging the Gap Between Saliency Prediction and Image Quality Assessment
• Authors: Kirillov Alexey, Andrey Moskalenko, Dmitriy Vatolin,
• Abstract summary: Deep neural models have made considerable advances in image quality assessment (IQA) We conduct an empirical study that reveals the relation between IQA and Saliency Prediction tasks. We introduce a novel SACID dataset of saliency-aware compressed images and conduct a large-scale comparison of classic and neural-based IQA methods.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Over the past few years, deep neural models have made considerable advances in image quality assessment (IQA). However, the underlying reasons for their success remain unclear, owing to the complex nature of deep neural networks. IQA aims to describe how the human visual system (HVS) works and to create its efficient approximations. On the other hand, Saliency Prediction task aims to emulate HVS via determining areas of visual interest. Thus, we believe that saliency plays a crucial role in human perception. In this work, we conduct an empirical study that reveals the relation between IQA and Saliency Prediction tasks, demonstrating that the former incorporates knowledge of the latter. Moreover, we introduce a novel SACID dataset of saliency-aware compressed images and conduct a large-scale comparison of classic and neural-based IQA methods. All supplementary code and data will be available at the time of publication.

Related papers

• Sliced Maximal Information Coefficient: A Training-Free Approach for Image Quality Assessment Enhancement [12.628718661568048]
  We aim to explore a generalized human visual attention estimation strategy to mimic the process of human quality rating. In particular, we model human attention generation by measuring the statistical dependency between the degraded image and the reference image. Experimental results verify the performance of existing IQA models can be consistently improved when our attention module is incorporated.
  arXiv  Detail & Related papers  (2024-08-19T11:55:32Z)
• Understanding and Evaluating Human Preferences for AI Generated Images with Instruction Tuning [58.41087653543607]
  We first establish a novel Image Quality Assessment (IQA) database for AIGIs, termed AIGCIQA2023+. This paper presents a MINT-IQA model to evaluate and explain human preferences for AIGIs from Multi-perspectives with INstruction Tuning.
  arXiv  Detail & Related papers  (2024-05-12T17:45:11Z)
• Large Multi-modality Model Assisted AI-Generated Image Quality Assessment [53.182136445844904]
  We introduce a large Multi-modality model Assisted AI-Generated Image Quality Assessment (MA-AGIQA) model. It uses semantically informed guidance to sense semantic information and extract semantic vectors through carefully designed text prompts. It achieves state-of-the-art performance, and demonstrates its superior generalization capabilities on assessing the quality of AI-generated images.
  arXiv  Detail & Related papers  (2024-04-27T02:40:36Z)
• Multi-Modal Prompt Learning on Blind Image Quality Assessment [65.0676908930946]
  Image Quality Assessment (IQA) models benefit significantly from semantic information, which allows them to treat different types of objects distinctly. Traditional methods, hindered by a lack of sufficiently annotated data, have employed the CLIP image-text pretraining model as their backbone to gain semantic awareness. Recent approaches have attempted to address this mismatch using prompt technology, but these solutions have shortcomings. This paper introduces an innovative multi-modal prompt-based methodology for IQA.
  arXiv  Detail & Related papers  (2024-04-23T11:45:32Z)
• Perceptual Attacks of No-Reference Image Quality Models with Human-in-the-Loop [113.75573175709573]
  We make one of the first attempts to examine the perceptual robustness of NR-IQA models. We test one knowledge-driven and three data-driven NR-IQA methods under four full-reference IQA models. We find that all four NR-IQA models are vulnerable to the proposed perceptual attack.
  arXiv  Detail & Related papers  (2022-10-03T13:47:16Z)
• Going the Extra Mile in Face Image Quality Assessment: A Novel Database and Model [42.05084438912876]
  We introduce the largest annotated IQA database developed to date, which contains 20,000 human faces. We propose a novel deep learning model to accurately predict face image quality, which, for the first time, explores the use of generative priors for IQA.
  arXiv  Detail & Related papers  (2022-07-11T14:28:18Z)
• Continual Learning for Blind Image Quality Assessment [80.55119990128419]
  Blind image quality assessment (BIQA) models fail to continually adapt to subpopulation shift. Recent work suggests training BIQA methods on the combination of all available human-rated IQA datasets. We formulate continual learning for BIQA, where a model learns continually from a stream of IQA datasets.
  arXiv  Detail & Related papers  (2021-02-19T03:07:01Z)
• Neuro-Symbolic Visual Reasoning: Disentangling "Visual" from "Reasoning" [49.76230210108583]
  We propose a framework to isolate and evaluate the reasoning aspect of visual question answering (VQA) separately from its perception. We also propose a novel top-down calibration technique that allows the model to answer reasoning questions even with imperfect perception. On the challenging GQA dataset, this framework is used to perform in-depth, disentangled comparisons between well-known VQA models.
  arXiv  Detail & Related papers  (2020-06-20T08:48:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.