How LLMs See Creativity: Zero-Shot Scoring of Visual Creativity with Interpretable Reasoning

• URL: http://arxiv.org/abs/2606.29672v2
• Date: Tue, 30 Jun 2026 11:29:35 GMT
• Title: How LLMs See Creativity: Zero-Shot Scoring of Visual Creativity with Interpretable Reasoning
• Abstract summary: multimodal large language models (LLMs) can serve as judges of visual creativity zero-shot.<n>Our findings indicate that multimodal LLMs can match human judgments of visual creativity without any additional training.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Evaluating the originality of visual images poses enduring challenges for creativity assessment. Automated scoring using AI models has proven effective in the verbal domain, yet key questions remain about evaluating visual creativity and understanding how models arrive at their ratings. The present research asks whether multimodal large language models (LLMs) can serve as judges of visual creativity zero-shot (without any fine-tuning or examples of human ratings) and whether their "reasoning" output offers an interpretable window into their evaluation process. We tested six multimodal LLMs (Gemini 3 Flash, Gemma 4 31B IT, GPT-5.4 Mini, GLM-5v Turbo, Kimi K2.5, and Qwen 3.6 Plus) on 992 AI-generated images (based on human-written prompts) and 1,500 hand-drawn sketches scored for creativity by human raters. In Study 1, all models showed substantial alignment with human creativity ratings on both datasets (r = .57-.68 on AI-generated images; r = .29-68 on sketches). In Study 2, we analyzed the step-by-step reasoning processes of three LLMs evaluating the same images and drawings. Although reasoning made model evaluations interpretable -- showing what they attend to, how they balance originality vs. quality, and how they justify their ratings -- reasoning did not improve alignment with human ratings. In sum, our findings indicate that multimodal LLMs can match human judgments of visual creativity without any additional training, and that their reasoning reveals how AI models evaluate creativity. An open scoring app implementing this pipeline is available at https://review-visual-eval-scoring.hf.space.

Related papers

• SketchVLM: Vision language models can annotate images to explain thoughts and guide users [16.25722200375932]
  We present SketchVLM, a training-free, model-agnostic framework that enables vision-language models to produce non-destructive, editable overlays on the input image to visually explain their answers.<n>We find that single-turn generation already achieves strong accuracy and annotation quality, and multi-turn generation opens up further opportunities for human-AI collaboration.
  arXiv  Detail & Related papers  (2026-04-23T22:33:15Z)
• Fine-Tuning a Large Vision-Language Model for Artwork's Scoring and Critique [11.787232686718367]
  We propose a framework for automated creativity assessment of human paintings by fine-tuning the vision-language model Qwen2-VL-7B with multi-task learning.<n>Our dataset contains 1000 human-created paintings scored on a 1-100 scale and paired with a short human-written description.<n> Experiments show strong accuracy, achieving Pearson r > 0.97 and MAE about 3.95 on the 100-point scale.
  arXiv  Detail & Related papers  (2026-02-09T19:52:16Z)
• MentisOculi: Revealing the Limits of Reasoning with Mental Imagery [63.285794947638614]
  We develop MentisOculi, a suite of multi-step reasoning problems amenable to visual solution.<n> evaluating visual strategies ranging from latent tokens to explicit generated imagery, we find they generally fail to improve performance.<n>Our findings suggest that despite their inherent appeal, visual thoughts do not yet benefit model reasoning.
  arXiv  Detail & Related papers  (2026-02-02T18:49:06Z)
• Simple Lines, Big Ideas: Towards Interpretable Assessment of Human Creativity from Drawings [18.09092203643732]
  We propose a data-driven framework for automatic and interpretable creativity assessment from drawings.<n>Motivated by the cognitive evidence proposed in [6] that creativity can emerge from both what is drawn (content) and how it is drawn (style), we reinterpret the creativity score as a function of these two complementary dimensions.
  arXiv  Detail & Related papers  (2025-11-17T02:16:01Z)
• LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models [59.0256377330646]
  Lens is a benchmark with 3.4K contemporary images and 60K+ human-authored questions covering eight tasks and 12 daily scenarios.<n>This dataset intrinsically supports to evaluate MLLMs to handle image-invariable prompts, from basic perception to compositional reasoning.<n>We evaluate 15+ frontier MLLMs such as Qwen2.5-VL-72B, InternVL3-78B, GPT-4o and two reasoning models QVQ-72B-preview and Kimi-VL.
  arXiv  Detail & Related papers  (2025-05-21T15:06:59Z)
• FLIP Reasoning Challenge [20.706469085872516]
  This paper introduces the FLIP dataset, a benchmark for evaluating AI reasoning capabilities based on human verification tasks.<n>FLIP challenges present users with two orderings of 4 images, requiring them to identify the coherent one.<n>Our experiments evaluate state-of-the-art models, leveraging both vision-language models (VLMs) and large language models (LLMs)
  arXiv  Detail & Related papers  (2025-04-16T17:07:16Z)
• GRADEO: Towards Human-Like Evaluation for Text-to-Video Generation via Multi-Step Reasoning [62.775721264492994]
  GRADEO is one of the first specifically designed video evaluation models.<n>It grades AI-generated videos for explainable scores and assessments through multi-step reasoning.<n> Experiments show that our method aligns better with human evaluations than existing methods.
  arXiv  Detail & Related papers  (2025-03-04T07:04:55Z)
• Leveraging Large Models to Evaluate Novel Content: A Case Study on Advertisement Creativity [25.460598990334077]
  We attempt to break down visual advertisement creativity into atypicality and originality.<n>We propose a suite of tasks specifically for such a subjective problem.<n>We also evaluate the alignment between state-of-the-art (SoTA) vision language models (VLMs) and humans on our proposed benchmark.
  arXiv  Detail & Related papers  (2025-02-26T04:28:03Z)
• Probing Visual Language Priors in VLMs [51.016683265437536]
  We introduce ViLP, a benchmark featuring deliberately out-of-distribution images.<n>Each question in ViLP is coupled with three potential answers and three corresponding images.<n>We propose a self-improving framework in which models generate new VQA data, then apply pixel-level and semantic corruptions to form "good-bad" image pairs for self-training.
  arXiv  Detail & Related papers  (2024-12-31T17:54:29Z)
• VHELM: A Holistic Evaluation of Vision Language Models [75.88987277686914]
  We present the Holistic Evaluation of Vision Language Models (VHELM) VHELM aggregates various datasets to cover one or more of the 9 aspects: visual perception, knowledge, reasoning, bias, fairness, multilinguality, robustness, toxicity, and safety. Our framework is designed to be lightweight and automatic so that evaluation runs are cheap and fast.
  arXiv  Detail & Related papers  (2024-10-09T17:46:34Z)
• GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generation [103.3465421081531]
  VQAScore is a metric measuring the likelihood that a VQA model views an image as accurately depicting the prompt. Ranking by VQAScore is 2x to 3x more effective than other scoring methods like PickScore, HPSv2, and ImageReward. We release a new GenAI-Rank benchmark with over 40,000 human ratings to evaluate scoring metrics on ranking images generated from the same prompt.
  arXiv  Detail & Related papers  (2024-06-19T18:00:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.