Image Quality Assessment: From Human to Machine Preference

• URL: http://arxiv.org/abs/2503.10078v1
• Date: Thu, 13 Mar 2025 05:58:38 GMT
• Title: Image Quality Assessment: From Human to Machine Preference
• Authors: Chunyi Li, Yuan Tian, Xiaoyue Ling, Zicheng Zhang, Haodong Duan, Haoning Wu, Ziheng Jia, Xiaohong Liu, Xiongkuo Min, Guo Lu, Weisi Lin, Guangtao Zhai,
• Abstract summary: This paper proposes the topic: Image Quality Assessment for Machine Vision.<n>We defined the subjective preferences of machines, including downstream tasks, test models, and evaluation metrics.<n>We also established the Machine Preference Database (MPD), which contains 2.25M fine-grained annotations and 30k reference/distorted image pair instances.
• Score: 88.01333947203132
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Image Quality Assessment (IQA) based on human subjective preferences has undergone extensive research in the past decades. However, with the development of communication protocols, the visual data consumption volume of machines has gradually surpassed that of humans. For machines, the preference depends on downstream tasks such as segmentation and detection, rather than visual appeal. Considering the huge gap between human and machine visual systems, this paper proposes the topic: Image Quality Assessment for Machine Vision for the first time. Specifically, we (1) defined the subjective preferences of machines, including downstream tasks, test models, and evaluation metrics; (2) established the Machine Preference Database (MPD), which contains 2.25M fine-grained annotations and 30k reference/distorted image pair instances; (3) verified the performance of mainstream IQA algorithms on MPD. Experiments show that current IQA metrics are human-centric and cannot accurately characterize machine preferences. We sincerely hope that MPD can promote the evolution of IQA from human to machine preferences. Project page is on: https://github.com/lcysyzxdxc/MPD.

Related papers

• What Are You Doing? A Closer Look at Controllable Human Video Generation [73.89117620413724]
  What Are You Doing?' is a new benchmark for evaluation of controllable image-to-video generation of humans.<n>It consists of 1,544 captioned videos that have been meticulously collected and annotated with 56 fine-grained categories.<n>We perform in-depth analyses of seven state-of-the-art models in controllable image-to-video generation.
  arXiv  Detail & Related papers  (2025-03-06T17:59:29Z)
• Embodied Image Quality Assessment for Robotic Intelligence [36.80460733311791]
  We first propose an embodied image quality assessment (EIQA) frameworks.<n>We establish assessment metrics for input images based on the downstream tasks of robot.<n>Experiments demonstrate that quality assessment of embodied images is different from that of humans.
  arXiv  Detail & Related papers  (2024-12-25T04:29:22Z)
• Quality Assessment for AI Generated Images with Instruction Tuning [58.41087653543607]
  We first establish a novel Image Quality Assessment (IQA) database for AIGIs, termed AIGCIQA2023+. This paper presents a MINT-IQA model to evaluate and explain human preferences for AIGIs from Multi-perspectives with INstruction Tuning.
  arXiv  Detail & Related papers  (2024-05-12T17:45:11Z)
• A Comparative Study of Perceptual Quality Metrics for Audio-driven Talking Head Videos [81.54357891748087]
  We collect talking head videos generated from four generative methods. We conduct controlled psychophysical experiments on visual quality, lip-audio synchronization, and head movement naturalness. Our experiments validate consistency between model predictions and human annotations, identifying metrics that align better with human opinions than widely-used measures.
  arXiv  Detail & Related papers  (2024-03-11T04:13:38Z)
• MMBench: Is Your Multi-modal Model an All-around Player? [114.45702807380415]
  We propose MMBench, a benchmark for assessing the multi-modal capabilities of vision-language models. MMBench is meticulously curated with well-designed quality control schemes. MMBench incorporates multiple-choice questions in both English and Chinese versions.
  arXiv  Detail & Related papers  (2023-07-12T16:23:09Z)
• AGIQA-3K: An Open Database for AI-Generated Image Quality Assessment [62.8834581626703]
  We build the most comprehensive subjective quality database AGIQA-3K so far. We conduct a benchmark experiment on this database to evaluate the consistency between the current Image Quality Assessment (IQA) model and human perception. We believe that the fine-grained subjective scores in AGIQA-3K will inspire subsequent AGI quality models to fit human subjective perception mechanisms.
  arXiv  Detail & Related papers  (2023-06-07T18:28:21Z)
• Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective [93.56647950778357]
  Blind image quality assessment (BIQA) predicts the human perception of image quality without any reference information. We develop a general and automated multitask learning scheme for BIQA to exploit auxiliary knowledge from other tasks.
  arXiv  Detail & Related papers  (2023-03-27T07:58:09Z)
• Going the Extra Mile in Face Image Quality Assessment: A Novel Database and Model [42.05084438912876]
  We introduce the largest annotated IQA database developed to date, which contains 20,000 human faces. We propose a novel deep learning model to accurately predict face image quality, which, for the first time, explores the use of generative priors for IQA.
  arXiv  Detail & Related papers  (2022-07-11T14:28:18Z)
• To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translation [5.972205906525993]
  We investigate which metrics have the highest accuracy to make system-level quality rankings for pairs of systems. We show that the sole use of BLEU negatively affected the past development of improved models.
  arXiv  Detail & Related papers  (2021-07-22T17:22:22Z)
• Partial success in closing the gap between human and machine vision [30.78663978510427]
  A few years ago, the first CNN surpassed human performance on ImageNet. Here we ask: Are we making progress in closing the gap between human and machine vision? We tested human observers on a broad range of out-of-distribution (OOD) datasets.
  arXiv  Detail & Related papers  (2021-06-14T13:23:35Z)
• Biometric Quality: Review and Application to Face Recognition with FaceQnet [16.791628175513637]
  Quality is the number one factor responsible for the good or bad performance of automated biometric systems. Some of the most used and deployed biometric characteristics are lacking behind in the development of these methods. FaceQnet is a novel open-source face quality assessment tool, inspired and powered by deep learning technology.
  arXiv  Detail & Related papers  (2020-06-05T08:33:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.