Image Aesthetics Assessment via Learnable Queries
- URL: http://arxiv.org/abs/2309.02861v1
- Date: Wed, 6 Sep 2023 09:42:16 GMT
- Title: Image Aesthetics Assessment via Learnable Queries
- Authors: Zhiwei Xiong, Yunfan Zhang, Zhiqi Shen, Peiran Ren, Han Yu
- Abstract summary: We propose the Image Aesthetics Assessment via Learnable Queries (IAA-LQ) approach.
It adapts learnable queries to extract aesthetic features from pre-trained image features obtained from a frozen image encoder.
Experiments on real-world data demonstrate the advantages of IAA-LQ, beating the best state-of-the-art method by 2.2% and 2.1% in terms of SRCC and PLCC, respectively.
- Score: 59.313054821874864
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Image aesthetics assessment (IAA) aims to estimate the aesthetics of images.
Depending on the content of an image, diverse criteria need to be selected to
assess its aesthetics. Existing works utilize pre-trained vision backbones
based on content knowledge to learn image aesthetics. However, training those
backbones is time-consuming and suffers from attention dispersion. Inspired by
learnable queries in vision-language alignment, we propose the Image Aesthetics
Assessment via Learnable Queries (IAA-LQ) approach. It adapts learnable queries
to extract aesthetic features from pre-trained image features obtained from a
frozen image encoder. Extensive experiments on real-world data demonstrate the
advantages of IAA-LQ, beating the best state-of-the-art method by 2.2% and 2.1%
in terms of SRCC and PLCC, respectively.
Related papers
- AID-AppEAL: Automatic Image Dataset and Algorithm for Content Appeal Enhancement and Assessment Labeling [11.996211235559866]
Image Content Appeal Assessment (ICAA) is a novel metric that quantifies the level of positive interest an image's content generates for viewers.
ICAA is different from traditional Image-Aesthetics Assessment (IAA), which judges an image's artistic quality.
arXiv Detail & Related papers (2024-07-08T01:40:32Z) - AACP: Aesthetics assessment of children's paintings based on
self-supervised learning [17.672268781368672]
The Aesthetics Assessment of Children's Paintings (AACP) is an important branch of the image aesthetics assessment (IAA)
Previous approaches have relied on training large datasets and providing an aesthetics score to the image.
We construct an aesthetics assessment dataset of children's paintings and a model based on self-supervised learning.
arXiv Detail & Related papers (2024-03-12T12:07:00Z) - VILA: Learning Image Aesthetics from User Comments with Vision-Language
Pretraining [53.470662123170555]
We propose learning image aesthetics from user comments, and exploring vision-language pretraining methods to learn multimodal aesthetic representations.
Specifically, we pretrain an image-text encoder-decoder model with image-comment pairs, using contrastive and generative objectives to learn rich and generic aesthetic semantics without human labels.
Our results show that our pretrained aesthetic vision-language model outperforms prior works on image aesthetic captioning over the AVA-Captions dataset.
arXiv Detail & Related papers (2023-03-24T23:57:28Z) - Aesthetic Visual Question Answering of Photographs [15.83390933825182]
We propose a new task of aesthetic language assessment: aesthetic visual question and answering (AVQA) of images.
The objective QA pairs are generated by the proposed aesthetic attributes analysis algorithms.
We build the first aesthetic visual question answering dataset, AesVQA, that contains 72,168 high-quality images and 324,756 pairs of aesthetic questions.
arXiv Detail & Related papers (2022-08-10T07:27:57Z) - Exploring CLIP for Assessing the Look and Feel of Images [87.97623543523858]
We introduce Contrastive Language-Image Pre-training (CLIP) models for assessing both the quality perception (look) and abstract perception (feel) of images in a zero-shot manner.
Our results show that CLIP captures meaningful priors that generalize well to different perceptual assessments.
arXiv Detail & Related papers (2022-07-25T17:58:16Z) - Distilling Knowledge from Object Classification to Aesthetics Assessment [68.317720070755]
The major dilemma of image aesthetics assessment (IAA) comes from the abstract nature of aesthetic labels.
We propose to distill knowledge on semantic patterns for a vast variety of image contents to an IAA model.
By supervising an end-to-end single-backbone IAA model with the distilled knowledge, the performance of the IAA model is significantly improved.
arXiv Detail & Related papers (2022-06-02T00:39:01Z) - Learning Conditional Knowledge Distillation for Degraded-Reference Image
Quality Assessment [157.1292674649519]
We propose a practical solution named degraded-reference IQA (DR-IQA)
DR-IQA exploits the inputs of IR models, degraded images, as references.
Our results can even be close to the performance of full-reference settings.
arXiv Detail & Related papers (2021-08-18T02:35:08Z) - User-Guided Personalized Image Aesthetic Assessment based on Deep
Reinforcement Learning [64.07820203919283]
We propose a novel user-guided personalized image aesthetic assessment framework.
It leverages user interactions to retouch and rank images for aesthetic assessment based on deep reinforcement learning (DRL)
It generates personalized aesthetic distribution that is more in line with the aesthetic preferences of different users.
arXiv Detail & Related papers (2021-06-14T15:19:48Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.