Enhancing Image Aesthetics with Dual-Conditioned Diffusion Models Guided by Multimodal Perception

• URL: http://arxiv.org/abs/2603.11556v1
• Date: Thu, 12 Mar 2026 05:22:53 GMT
• Title: Enhancing Image Aesthetics with Dual-Conditioned Diffusion Models Guided by Multimodal Perception
• Abstract summary: Image aesthetic enhancement aims to perceive aesthetic deficiencies in images and perform corresponding editing operations.<n>Recent advancements in image editing models have significantly enhanced their controllability and flexibility, but they struggle with enhancing image aesthetic.<n>We propose Dual-supervised Image Aesthetic Enhancement (DIAE), a diffusion-based generative model with multimodal aesthetic perception.
• Score: 6.873293280691424
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image aesthetic enhancement aims to perceive aesthetic deficiencies in images and perform corresponding editing operations, which is highly challenging and requires the model to possess creativity and aesthetic perception capabilities. Although recent advancements in image editing models have significantly enhanced their controllability and flexibility, they struggle with enhancing image aesthetic. The primary challenges are twofold: first, following editing instructions with aesthetic perception is difficult, and second, there is a scarcity of "perfectly-paired" images that have consistent content but distinct aesthetic qualities. In this paper, we propose Dual-supervised Image Aesthetic Enhancement (DIAE), a diffusion-based generative model with multimodal aesthetic perception. First, DIAE incorporates Multimodal Aesthetic Perception (MAP) to convert the ambiguous aesthetic instruction into explicit guidance by (i) employing detailed, standardized aesthetic instructions across multiple aesthetic attributes, and (ii) utilizing multimodal control signals derived from text-image pairs that maintain consistency within the same aesthetic attribute. Second, to mitigate the lack of "perfectly-paired" images, we collect "imperfectly-paired" dataset called IIAEData, consisting of images with varying aesthetic qualities while sharing identical semantics. To better leverage the weak matching characteristics of IIAEData during training, a dual-branch supervision framework is also introduced for weakly supervised image aesthetic enhancement. Experimental results demonstrate that DIAE outperforms the baselines and obtains superior image aesthetic scores and image content consistency scores.

Related papers

• Venus: Benchmarking and Empowering Multimodal Large Language Models for Aesthetic Guidance and Cropping [47.103757942619914]
  Smartphones have made photography ubiquitous, yet a clear gap remains between ordinary users and professional photographers.<n>We define aesthetic guidance (AG) as an essential but largely underexplored domain in computational aesthetics.<n>We introduce AesGuide, the first large-scale AG dataset and benchmark with 10,748 photos annotated with aesthetic scores, analyses, and guidance.<n>We propose Venus, a two-stage framework that first empowers MLLMs with AG capability through progressively complex aesthetic questions.
  arXiv  Detail & Related papers  (2026-02-27T12:47:31Z)
• Bridging Cognitive Gap: Hierarchical Description Learning for Artistic Image Aesthetics Assessment [51.40989269202702]
  aesthetic quality assessment task is crucial for developing a human-aligned quantitative evaluation system for AIGC.<n>We propose ArtQuant, an aesthetics assessment framework for artistic images which couples isolated aesthetic dimensions through description generation.<n>Our approach achieves epoch state-of-the-art performance on several datasets while requiring only 33% of conventional trainings.
  arXiv  Detail & Related papers  (2025-12-29T12:18:26Z)
• Compose Your Aesthetics: Empowering Text-to-Image Models with the Principles of Art [61.28133495240179]
  We propose a novel task of aesthetics alignment which seeks to align user-specified aesthetics with the T2I generation output.<n>Inspired by how artworks provide an invaluable perspective to approach aesthetics, we codify visual aesthetics using the compositional framework artists employ.<n>We demonstrate that T2I DMs can effectively offer 10 compositional controls through user-specified PoA conditions.
  arXiv  Detail & Related papers  (2025-03-15T06:58:09Z)
• Advancing Comprehensive Aesthetic Insight with Multi-Scale Text-Guided Self-Supervised Learning [14.405750888492735]
  Image Aesthetic Assessment (IAA) is a vital and intricate task that entails analyzing and assessing an image's aesthetic values.<n>Traditional methods of IAA often concentrate on a single aesthetic task and suffer from inadequate labeled datasets.<n>We propose a comprehensive aesthetic MLLM capable of nuanced aesthetic insight.
  arXiv  Detail & Related papers  (2024-12-16T16:35:35Z)
• AesExpert: Towards Multi-modality Foundation Model for Image Aesthetics Perception [74.11069437400398]
  We develop a corpus-rich aesthetic critique database with 21,904 diverse-sourced images and 88K human natural language feedbacks. We fine-tune the open-sourced general foundation models, achieving multi-modality Aesthetic Expert models, dubbed AesExpert. Experiments demonstrate that the proposed AesExpert models deliver significantly better aesthetic perception performances than the state-of-the-art MLLMs.
  arXiv  Detail & Related papers  (2024-04-15T09:56:20Z)
• Image Aesthetics Assessment via Learnable Queries [59.313054821874864]
  We propose the Image Aesthetics Assessment via Learnable Queries (IAA-LQ) approach. It adapts learnable queries to extract aesthetic features from pre-trained image features obtained from a frozen image encoder. Experiments on real-world data demonstrate the advantages of IAA-LQ, beating the best state-of-the-art method by 2.2% and 2.1% in terms of SRCC and PLCC, respectively.
  arXiv  Detail & Related papers  (2023-09-06T09:42:16Z)
• VILA: Learning Image Aesthetics from User Comments with Vision-Language Pretraining [53.470662123170555]
  We propose learning image aesthetics from user comments, and exploring vision-language pretraining methods to learn multimodal aesthetic representations. Specifically, we pretrain an image-text encoder-decoder model with image-comment pairs, using contrastive and generative objectives to learn rich and generic aesthetic semantics without human labels. Our results show that our pretrained aesthetic vision-language model outperforms prior works on image aesthetic captioning over the AVA-Captions dataset.
  arXiv  Detail & Related papers  (2023-03-24T23:57:28Z)
• User-Guided Personalized Image Aesthetic Assessment based on Deep Reinforcement Learning [64.07820203919283]
  We propose a novel user-guided personalized image aesthetic assessment framework. It leverages user interactions to retouch and rank images for aesthetic assessment based on deep reinforcement learning (DRL) It generates personalized aesthetic distribution that is more in line with the aesthetic preferences of different users.
  arXiv  Detail & Related papers  (2021-06-14T15:19:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.