What Do Vision-Language Models Encode for Personalized Image Aesthetics Assessment?

• URL: http://arxiv.org/abs/2604.11374v1
• Date: Mon, 13 Apr 2026 12:15:24 GMT
• Title: What Do Vision-Language Models Encode for Personalized Image Aesthetics Assessment?
• Abstract summary: We analyze vision-language models (VLMs) to examine the presence and distribution of rich, multi-level aesthetic attributes.<n>Our analysis reveals that VLMs encode diverse aesthetic attributes that propagate into the language decoder layers.<n>Our findings provide insights into how VLMs can be utilized for modeling subjective, individual aesthetic preferences.
• Score: 16.394933051332657
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Personalized image aesthetics assessment (PIAA) is an important research problem with practical real-world applications. While methods based on vision-language models (VLMs) are promising candidates for PIAA, it remains unclear whether they internally encode rich, multi-level aesthetic attributes required for effective personalization. In this paper, we first analyze the internal representations of VLMs to examine the presence and distribution of such aesthetic attributes, and then leverage them for lightweight, individual-level personalization without model fine-tuning. Our analysis reveals that VLMs encode diverse aesthetic attributes that propagate into the language decoder layers. Building on these representations, we demonstrate that simple linear models can perform PIAA effectively. We further analyze how aesthetic information is transferred across layers in different VLM architectures and across image domains. Our findings provide insights into how VLMs can be utilized for modeling subjective, individual aesthetic preferences. Our code is available at https://github.com/ynklab/vlm-latent-piaa.

Related papers

• Attention Guided Alignment in Efficient Vision-Language Models [56.20286899428444]
  Large Vision-Language Models (VLMs) rely on effective multimodal alignment between pre-trained vision encoders and Large Language Models (LLMs)<n>This paper presents a comprehensive analysis of attention patterns in efficient VLMs.<n>We introduce Attention-Guided Efficient Vision-Language Models (AGE-VLM), a novel framework that enhances visual grounding through interleaved cross-attention layers.
  arXiv  Detail & Related papers  (2025-11-21T21:36:48Z)
• Reading Images Like Texts: Sequential Image Understanding in Vision-Language Models [9.24989979549793]
  Vision-Language Models (VLMs) have demonstrated remarkable performance across a variety of real-world tasks.<n>These models typically process visual information by serializing images, a method that diverges significantly from the parallel nature of human vision.<n>We introduce an instruction-agnostic token compression algorithm based on a plug-and-play visual decoder to improve decoding efficiency.
  arXiv  Detail & Related papers  (2025-09-23T16:07:18Z)
• Advancing Comprehensive Aesthetic Insight with Multi-Scale Text-Guided Self-Supervised Learning [14.405750888492735]
  Image Aesthetic Assessment (IAA) is a vital and intricate task that entails analyzing and assessing an image's aesthetic values.<n>Traditional methods of IAA often concentrate on a single aesthetic task and suffer from inadequate labeled datasets.<n>We propose a comprehensive aesthetic MLLM capable of nuanced aesthetic insight.
  arXiv  Detail & Related papers  (2024-12-16T16:35:35Z)
• Elevating Visual Perception in Multimodal LLMs with Visual Embedding Distillation [109.5893580175657]
  In recent times, the standard practice for developing MLLMs is to feed features from vision encoder(s) into the LLM and train with natural language supervision.<n>This approach often causes models to lean towards language comprehension and undermine the rich visual perception signals present in the data.<n>We propose VisPer-LM, the first approach that infuses visual perception knowledge from expert vision encoders into the LLM's hidden representations.
  arXiv  Detail & Related papers  (2024-12-12T18:55:18Z)
• Aligning Vision Models with Human Aesthetics in Retrieval: Benchmarks and Algorithms [91.19304518033144]
  We aim to align vision models with human aesthetic standards in a retrieval system. We propose a preference-based reinforcement learning method that fine-tunes the vision models to better align the vision models with human aesthetics.
  arXiv  Detail & Related papers  (2024-06-13T17:59:20Z)
• AesExpert: Towards Multi-modality Foundation Model for Image Aesthetics Perception [74.11069437400398]
  We develop a corpus-rich aesthetic critique database with 21,904 diverse-sourced images and 88K human natural language feedbacks. We fine-tune the open-sourced general foundation models, achieving multi-modality Aesthetic Expert models, dubbed AesExpert. Experiments demonstrate that the proposed AesExpert models deliver significantly better aesthetic perception performances than the state-of-the-art MLLMs.
  arXiv  Detail & Related papers  (2024-04-15T09:56:20Z)
• Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models [73.40350756742231]
  Visually-conditioned language models (VLMs) have seen growing adoption in applications such as visual dialogue, scene understanding, and robotic task planning. Despite the volume of new releases, key design decisions around image preprocessing, architecture, and optimization are under-explored.
  arXiv  Detail & Related papers  (2024-02-12T18:21:14Z)
• Unveiling The Factors of Aesthetic Preferences with Explainable AI [0.0]
  In this study, we pioneer a novel perspective by utilizing several different machine learning (ML) models. Our models process these attributes as inputs to predict the aesthetic scores of images. Our aim is to shed light on the complex nature of aesthetic preferences in images through ML and to provide a deeper understanding of the attributes that influence aesthetic judgements.
  arXiv  Detail & Related papers  (2023-11-24T11:06:22Z)
• VILA: Learning Image Aesthetics from User Comments with Vision-Language Pretraining [53.470662123170555]
  We propose learning image aesthetics from user comments, and exploring vision-language pretraining methods to learn multimodal aesthetic representations. Specifically, we pretrain an image-text encoder-decoder model with image-comment pairs, using contrastive and generative objectives to learn rich and generic aesthetic semantics without human labels. Our results show that our pretrained aesthetic vision-language model outperforms prior works on image aesthetic captioning over the AVA-Captions dataset.
  arXiv  Detail & Related papers  (2023-03-24T23:57:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.