Considering user agreement in learning to predict the aesthetic quality

• URL: http://arxiv.org/abs/2110.06956v1
• Date: Wed, 13 Oct 2021 18:00:36 GMT
• Title: Considering user agreement in learning to predict the aesthetic quality
• Abstract summary: We propose a re-adapted multi-task attention network to predict both the mean opinion score and the standard deviation in an end-to-end manner. With such loss, the model is encouraged to learn the uncertainty of the content that is relevant to the diversity of observers' opinions. Experiments have demonstrated that the proposed multi-task aesthetic model achieves state-of-the-art performance on two different types of aesthetic datasets.
• Score: 35.255447771350404
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: How to robustly rank the aesthetic quality of given images has been a long-standing ill-posed topic. Such challenge stems mainly from the diverse subjective opinions of different observers about the varied types of content. There is a growing interest in estimating the user agreement by considering the standard deviation of the scores, instead of only predicting the mean aesthetic opinion score. Nevertheless, when comparing a pair of contents, few studies consider how confident are we regarding the difference in the aesthetic scores. In this paper, we thus propose (1) a re-adapted multi-task attention network to predict both the mean opinion score and the standard deviation in an end-to-end manner; (2) a brand-new confidence interval ranking loss that encourages the model to focus on image-pairs that are less certain about the difference of their aesthetic scores. With such loss, the model is encouraged to learn the uncertainty of the content that is relevant to the diversity of observers' opinions, i.e., user disagreement. Extensive experiments have demonstrated that the proposed multi-task aesthetic model achieves state-of-the-art performance on two different types of aesthetic datasets, i.e., AVA and TMGA.

Related papers

• Walkable to Whom? Capturing Subjective Variability in Walkability Perception Using Multimodal Deep Learning [0.12744523252873352]
  This paper introduces a dataset of 29,870 walkability ratings from 1,196 respondents in Australia.<n>It proposes the first user-conditioned multimodal deep learning framework for walkability perception.
  arXiv  Detail & Related papers  (2026-08-07T08:07:06Z)
• Beyond Absolute Scores: Relative Edit-induced Difference for Generalizable Image Aesthetic Assessment [7.510816222411201]
  We propose Relative Edit-induced Difference Aesthetic learning (RED-Aes) to simulate the human aesthetic reasoning process.<n>Instead of fitting absolute score distributions, RED-Aes explicitly learns the visual factors that drive aesthetic changes.<n>Experiments demonstrate that RED-Aes achieves state-of-the-art performance on multiple public benchmarks, exhibiting superior generalization capabilities.
  arXiv  Detail & Related papers  (2026-06-04T07:07:01Z)
• Not All Subjectivity Is the Same! Defining Desiderata for the Evaluation of Subjectivity in NLP [8.416822614574368]
  Subjective judgments are part of several NLP datasets.<n>Recent work is increasingly prioritizing models whose outputs reflect this diversity of perspectives.<n>It remains a question whether our evaluation practices align with these models' objectives.
  arXiv  Detail & Related papers  (2026-03-30T12:21:32Z)
• AesBiasBench: Evaluating Bias and Alignment in Multimodal Language Models for Personalized Image Aesthetic Assessment [29.2617518199559]
  AesBiasBench is a benchmark designed to evaluate Multimodal Large Language Models (MLLMs)<n>Results indicate that smaller models exhibit stronger stereotype biases, whereas larger models align more closely with human preferences.<n>These findings underscore the importance of identity-aware evaluation frameworks in subjective vision-language tasks.
  arXiv  Detail & Related papers  (2025-09-15T06:25:39Z)
• A Meaningful Perturbation Metric for Evaluating Explainability Methods [55.09730499143998]
  We introduce a novel approach, which harnesses image generation models to perform targeted perturbation. Specifically, we focus on inpainting only the high-relevance pixels of an input image to modify the model's predictions while preserving image fidelity. This is in contrast to existing approaches, which often produce out-of-distribution modifications, leading to unreliable results.
  arXiv  Detail & Related papers  (2025-04-09T11:46:41Z)
• Multi-Perspective Stance Detection [2.8073184910275293]
  Multi-perspective approach yields better classification performance than the baseline which uses the single label. This entails that designing more inclusive perspective-aware AI models is not only an essential first step in implementing responsible and ethical AI, but it can also achieve superior results than using the traditional approaches.
  arXiv  Detail & Related papers  (2024-11-13T16:30:41Z)
• When Does Perceptual Alignment Benefit Vision Representations? [76.32336818860965]
  We investigate how aligning vision model representations to human perceptual judgments impacts their usability. We find that aligning models to perceptual judgments yields representations that improve upon the original backbones across many downstream tasks. Our results suggest that injecting an inductive bias about human perceptual knowledge into vision models can contribute to better representations.
  arXiv  Detail & Related papers  (2024-10-14T17:59:58Z)
• Benchmarking the Attribution Quality of Vision Models [13.255247017616687]
  We propose a novel evaluation protocol that overcomes two fundamental limitations of the widely used incremental-deletion protocol. This allows us to evaluate 23 attribution methods and how different design choices of popular vision backbones affect their attribution quality. We find that intrinsically explainable models outperform standard models and that raw attribution values exhibit a higher attribution quality than what is known from previous work.
  arXiv  Detail & Related papers  (2024-07-16T17:02:20Z)
• Multi-Modal Prompt Learning on Blind Image Quality Assessment [65.0676908930946]
  Image Quality Assessment (IQA) models benefit significantly from semantic information, which allows them to treat different types of objects distinctly. Traditional methods, hindered by a lack of sufficiently annotated data, have employed the CLIP image-text pretraining model as their backbone to gain semantic awareness. Recent approaches have attempted to address this mismatch using prompt technology, but these solutions have shortcomings. This paper introduces an innovative multi-modal prompt-based methodology for IQA.
  arXiv  Detail & Related papers  (2024-04-23T11:45:32Z)
• Evaluating the Fairness of Discriminative Foundation Models in Computer Vision [51.176061115977774]
  We propose a novel taxonomy for bias evaluation of discriminative foundation models, such as Contrastive Language-Pretraining (CLIP) We then systematically evaluate existing methods for mitigating bias in these models with respect to our taxonomy. Specifically, we evaluate OpenAI's CLIP and OpenCLIP models for key applications, such as zero-shot classification, image retrieval and image captioning.
  arXiv  Detail & Related papers  (2023-10-18T10:32:39Z)
• Towards Reliable Assessments of Demographic Disparities in Multi-Label Image Classifiers [11.973749734226852]
  We consider multi-label image classification and, specifically, object categorization tasks. Design choices and trade-offs for measurement involve more nuance than discussed in prior computer vision literature. We identify several design choices that look merely like implementation details but significantly impact the conclusions of assessments.
  arXiv  Detail & Related papers  (2023-02-16T20:34:54Z)
• Understanding Aesthetics with Language: A Photo Critique Dataset for Aesthetic Assessment [6.201485014848172]
  We propose the Critique Photo Reddit dataset (RPCD), which contains 74K images and 220K comments. We exploit the polarity of the sentiment of criticism as an indicator of aesthetic judgment.
  arXiv  Detail & Related papers  (2022-06-17T08:16:20Z)
• Bongard-HOI: Benchmarking Few-Shot Visual Reasoning for Human-Object Interactions [138.49522643425334]
  Bongard-HOI is a new visual reasoning benchmark that focuses on compositional learning of human-object interactions from natural images. It is inspired by two desirable characteristics from the classical Bongard problems (BPs): 1) few-shot concept learning, and 2) context-dependent reasoning. Bongard-HOI presents a substantial challenge to today's visual recognition models.
  arXiv  Detail & Related papers  (2022-05-27T07:36:29Z)
• Uncertainty-Aware Few-Shot Image Classification [118.72423376789062]
  Few-shot image classification learns to recognize new categories from limited labelled data. We propose Uncertainty-Aware Few-Shot framework for image classification.
  arXiv  Detail & Related papers  (2020-10-09T12:26:27Z)
• Difficulty-aware Glaucoma Classification with Multi-Rater Consensus Modeling [34.28252351672568]
  We take advantage of the raw multi-rater gradings to improve the deep learning model performance for the glaucoma classification task. A multi-branch model structure is proposed to predict the most sensitive, most specifical and a balanced fused result for the input images. Compared with models trained only with the final ground-truth labels, the proposed method using multi-rater consensus information has achieved superior performance.
  arXiv  Detail & Related papers  (2020-07-29T14:04:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.