TIER: Text-Image Encoder-based Regression for AIGC Image Quality
Assessment
- URL: http://arxiv.org/abs/2401.03854v2
- Date: Thu, 11 Jan 2024 08:09:33 GMT
- Title: TIER: Text-Image Encoder-based Regression for AIGC Image Quality
Assessment
- Authors: Jiquan Yuan, Xinyan Cao, Jinming Che, Qinyuan Wang, Sen Liang, Wei
Ren, Jinlong Lin, Xixin Cao
- Abstract summary: In AIGCIQA tasks, images are typically generated by generative models using text prompts.
Most existing AIGCIQA methods regress predicted scores directly from individual generated images.
We propose a text-image encoder-based regression (TIER) framework to address this issue.
- Score: 2.59079758388817
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Recently, AIGC image quality assessment (AIGCIQA), which aims to assess the
quality of AI-generated images (AIGIs) from a human perception perspective, has
emerged as a new topic in computer vision. Unlike common image quality
assessment tasks where images are derived from original ones distorted by
noise, blur, and compression, \textit{etc.}, in AIGCIQA tasks, images are
typically generated by generative models using text prompts. Considerable
efforts have been made in the past years to advance AIGCIQA. However, most
existing AIGCIQA methods regress predicted scores directly from individual
generated images, overlooking the information contained in the text prompts of
these images. This oversight partially limits the performance of these AIGCIQA
methods. To address this issue, we propose a text-image encoder-based
regression (TIER) framework. Specifically, we process the generated images and
their corresponding text prompts as inputs, utilizing a text encoder and an
image encoder to extract features from these text prompts and generated images,
respectively. To demonstrate the effectiveness of our proposed TIER method, we
conduct extensive experiments on several mainstream AIGCIQA databases,
including AGIQA-1K, AGIQA-3K, and AIGCIQA2023. The experimental results
indicate that our proposed TIER method generally demonstrates superior
performance compared to baseline in most cases.
Related papers
- AI-generated Image Quality Assessment in Visual Communication [72.11144790293086]
AIGI-VC is a quality assessment database for AI-generated images in visual communication.
The dataset consists of 2,500 images spanning 14 advertisement topics and 8 emotion types.
It provides coarse-grained human preference annotations and fine-grained preference descriptions, benchmarking the abilities of IQA methods in preference prediction, interpretation, and reasoning.
arXiv Detail & Related papers (2024-12-20T08:47:07Z) - A Sanity Check for AI-generated Image Detection [49.08585395873425]
We propose AIDE (AI-generated Image DEtector with Hybrid Features) to detect AI-generated images.
AIDE achieves +3.5% and +4.6% improvements to state-of-the-art methods.
arXiv Detail & Related papers (2024-06-27T17:59:49Z) - Vision-Language Consistency Guided Multi-modal Prompt Learning for Blind AI Generated Image Quality Assessment [57.07360640784803]
We propose vision-language consistency guided multi-modal prompt learning for blind image quality assessment (AGIQA)
Specifically, we introduce learnable textual and visual prompts in language and vision branches of Contrastive Language-Image Pre-training (CLIP) models.
We design a text-to-image alignment quality prediction task, whose learned vision-language consistency knowledge is used to guide the optimization of the above multi-modal prompts.
arXiv Detail & Related papers (2024-06-24T13:45:31Z) - RIGID: A Training-free and Model-Agnostic Framework for Robust AI-Generated Image Detection [60.960988614701414]
RIGID is a training-free and model-agnostic method for robust AI-generated image detection.
RIGID significantly outperforms existing trainingbased and training-free detectors.
arXiv Detail & Related papers (2024-05-30T14:49:54Z) - Quality Assessment for AI Generated Images with Instruction Tuning [58.41087653543607]
We first establish a novel Image Quality Assessment (IQA) database for AIGIs, termed AIGCIQA2023+.
This paper presents a MINT-IQA model to evaluate and explain human preferences for AIGIs from Multi-perspectives with INstruction Tuning.
arXiv Detail & Related papers (2024-05-12T17:45:11Z) - PKU-AIGIQA-4K: A Perceptual Quality Assessment Database for Both Text-to-Image and Image-to-Image AI-Generated Images [1.5265677582796984]
We establish a large scale perceptual quality assessment database for both text-to-image and image-to-image AIGIs, named PKU-AIGIQA-4K.
We propose three image quality assessment (IQA) methods based on pre-trained models that include a no-reference method NR-AIGCIQA, a full-reference method FR-AIGCIQA, and a partial-reference method PR-AIGCIQA.
arXiv Detail & Related papers (2024-04-29T03:57:43Z) - AIGCOIQA2024: Perceptual Quality Assessment of AI Generated Omnidirectional Images [70.42666704072964]
We establish a large-scale AI generated omnidirectional image IQA database named AIGCOIQA2024.
A subjective IQA experiment is conducted to assess human visual preferences from three perspectives.
We conduct a benchmark experiment to evaluate the performance of state-of-the-art IQA models on our database.
arXiv Detail & Related papers (2024-04-01T10:08:23Z) - PSCR: Patches Sampling-based Contrastive Regression for AIGC Image
Quality Assessment [1.1744028458220428]
We propose a contrastive regression framework to leverage differences among various generated images for learning a better representation space.
We conduct extensive experiments on three mainstream AIGCIQA databases including AGIQA-1K, AGIQA-3K and AIGCIQA2023.
Results show significant improvements in model performance with the introduction of our proposed PSCR framework.
arXiv Detail & Related papers (2023-12-10T14:18:53Z) - PKU-I2IQA: An Image-to-Image Quality Assessment Database for AI
Generated Images [1.6031185986328562]
We establish a human perception-based image-to-image AIGCIQA database, named PKU-I2IQA.
We propose two benchmark models: NR-AIGCIQA based on the no-reference image quality assessment method and FR-AIGCIQA based on the full-reference image quality assessment method.
arXiv Detail & Related papers (2023-11-27T05:53:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.