AGIQA-3K: An Open Database for AI-Generated Image Quality Assessment
- URL: http://arxiv.org/abs/2306.04717v2
- Date: Mon, 12 Jun 2023 16:42:59 GMT
- Title: AGIQA-3K: An Open Database for AI-Generated Image Quality Assessment
- Authors: Chunyi Li, Zicheng Zhang, Haoning Wu, Wei Sun, Xiongkuo Min, Xiaohong
Liu, Guangtao Zhai, Weisi Lin
- Abstract summary: We build the most comprehensive subjective quality database AGIQA-3K so far.
We conduct a benchmark experiment on this database to evaluate the consistency between the current Image Quality Assessment (IQA) model and human perception.
We believe that the fine-grained subjective scores in AGIQA-3K will inspire subsequent AGI quality models to fit human subjective perception mechanisms.
- Score: 62.8834581626703
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: With the rapid advancements of the text-to-image generative model,
AI-generated images (AGIs) have been widely applied to entertainment,
education, social media, etc. However, considering the large quality variance
among different AGIs, there is an urgent need for quality models that are
consistent with human subjective ratings. To address this issue, we extensively
consider various popular AGI models, generated AGI through different prompts
and model parameters, and collected subjective scores at the perceptual quality
and text-to-image alignment, thus building the most comprehensive AGI
subjective quality database AGIQA-3K so far. Furthermore, we conduct a
benchmark experiment on this database to evaluate the consistency between the
current Image Quality Assessment (IQA) model and human perception, while
proposing StairReward that significantly improves the assessment performance of
subjective text-to-image alignment. We believe that the fine-grained subjective
scores in AGIQA-3K will inspire subsequent AGI quality models to fit human
subjective perception mechanisms at both perception and alignment levels and to
optimize the generation result of future AGI models. The database is released
on https://github.com/lcysyzxdxc/AGIQA-3k-Database.
Related papers
- Understanding and Evaluating Human Preferences for AI Generated Images with Instruction Tuning [58.41087653543607]
We first establish a novel Image Quality Assessment (IQA) database for AIGIs, termed AIGCIQA2023+.
This paper presents a MINT-IQA model to evaluate and explain human preferences for AIGIs from Multi-perspectives with INstruction Tuning.
arXiv Detail & Related papers (2024-05-12T17:45:11Z) - PKU-AIGIQA-4K: A Perceptual Quality Assessment Database for Both Text-to-Image and Image-to-Image AI-Generated Images [1.5265677582796984]
We establish a large scale perceptual quality assessment database for both text-to-image and image-to-image AIGIs, named PKU-AIGIQA-4K.
We propose three image quality assessment (IQA) methods based on pre-trained models that include a no-reference method NR-AIGCIQA, a full-reference method FR-AIGCIQA, and a partial-reference method PR-AIGCIQA.
arXiv Detail & Related papers (2024-04-29T03:57:43Z) - Large Multi-modality Model Assisted AI-Generated Image Quality Assessment [53.182136445844904]
We introduce a large Multi-modality model Assisted AI-Generated Image Quality Assessment (MA-AGIQA) model.
It uses semantically informed guidance to sense semantic information and extract semantic vectors through carefully designed text prompts.
It achieves state-of-the-art performance, and demonstrates its superior generalization capabilities on assessing the quality of AI-generated images.
arXiv Detail & Related papers (2024-04-27T02:40:36Z) - AIGIQA-20K: A Large Database for AI-Generated Image Quality Assessment [54.93996119324928]
We create the largest AIGI subjective quality database to date with 20,000 AIGIs and 420,000 subjective ratings, known as AIGIQA-20K.
We conduct benchmark experiments on this database to assess the correspondence between 16 mainstream AIGI quality models and human perception.
arXiv Detail & Related papers (2024-04-04T12:12:24Z) - AIGCOIQA2024: Perceptual Quality Assessment of AI Generated Omnidirectional Images [70.42666704072964]
We establish a large-scale AI generated omnidirectional image IQA database named AIGCOIQA2024.
A subjective IQA experiment is conducted to assess human visual preferences from three perspectives.
We conduct a benchmark experiment to evaluate the performance of state-of-the-art IQA models on our database.
arXiv Detail & Related papers (2024-04-01T10:08:23Z) - PKU-I2IQA: An Image-to-Image Quality Assessment Database for AI
Generated Images [1.6031185986328562]
We establish a human perception-based image-to-image AIGCIQA database, named PKU-I2IQA.
We propose two benchmark models: NR-AIGCIQA based on the no-reference image quality assessment method and FR-AIGCIQA based on the full-reference image quality assessment method.
arXiv Detail & Related papers (2023-11-27T05:53:03Z) - A Perceptual Quality Assessment Exploration for AIGC Images [39.72512063793346]
In this paper, we discuss the major evaluation aspects such as technical issues, AI artifacts, unnaturalness, discrepancy, and aesthetics for AGI quality assessment.
We present the first perceptual AGI quality assessment database, AGIQA-1K, which consists of 1,080 AGIs generated from diffusion models.
arXiv Detail & Related papers (2023-03-22T14:59:49Z) - Subjective and Objective Quality Assessment for in-the-Wild Computer
Graphics Images [57.02760260360728]
We build a large-scale in-the-wild CGIQA database consisting of 6,000 CGIs (CGIQA-6k)
We propose an effective deep learning-based no-reference (NR) IQA model by utilizing both distortion and aesthetic quality representation.
Experimental results show that the proposed method outperforms all other state-of-the-art NR IQA methods on the constructed CGIQA-6k database.
arXiv Detail & Related papers (2023-03-14T16:32:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.