Leveraging Large Models for Evaluating Novel Content: A Case Study on Advertisement Creativity

• URL: http://arxiv.org/abs/2503.00046v1
• Date: Wed, 26 Feb 2025 04:28:03 GMT
• Title: Leveraging Large Models for Evaluating Novel Content: A Case Study on Advertisement Creativity
• Authors: Zhaoyi Joey Hou, Adriana Kovashka, Xiang Lorraine Li,
• Abstract summary: We attempt to break down visual advertisement creativity into atypicality and originality.<n>With fine-grained human annotations, we propose a suit of tasks specifically for such a subjective problem.<n>We also evaluate the alignment between state-of-the-art (SoTA) vision language models (VLM) and humans on our proposed benchmark.
• Score: 26.90276644134837
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Evaluating creativity is challenging, even for humans, not only because of its subjectivity but also because it involves complex cognitive processes. Inspired by work in marketing, we attempt to break down visual advertisement creativity into atypicality and originality. With fine-grained human annotations on these dimensions, we propose a suit of tasks specifically for such a subjective problem. We also evaluate the alignment between state-of-the-art (SoTA) vision language models (VLM) and humans on our proposed benchmark, demonstrating both the promises and challenges of using VLMs for automatic creativity assessment.

Related papers

• Cooking Up Creativity: A Cognitively-Inspired Approach for Enhancing LLM Creativity through Structured Representations [53.950760059792614]
  Large Language Models (LLMs) excel at countless tasks, yet struggle with creativity. We introduce a novel approach that couples LLMs with structured representations and cognitively inspired manipulations to generate more creative and diverse ideas. We demonstrate our approach in the culinary domain with DishCOVER, a model that generates creative recipes.
  arXiv  Detail & Related papers  (2025-04-29T11:13:06Z)
• Probing and Inducing Combinational Creativity in Vision-Language Models [52.76981145923602]
  Recent advances in Vision-Language Models (VLMs) have sparked debate about whether their outputs reflect combinational creativity. We propose the Identification-Explanation-Implication (IEI) framework, which decomposes creative processes into three levels. To validate this framework, we curate CreativeMashup, a high-quality dataset of 666 artist-generated visual mashups annotated according to the IEI framework.
  arXiv  Detail & Related papers  (2025-04-17T17:38:18Z)
• Mitigating Low-Level Visual Hallucinations Requires Self-Awareness: Database, Model and Training Strategy [53.07517728420411]
  We introduce the first instruction database specifically focused on hallucinations in low-level vision tasks. We propose the Self-Awareness Failure Elimination (SAFEQA) model to improve the perception and comprehension abilities of the model in low-level vision tasks. We conduct comprehensive experiments on low-level vision tasks, with the results demonstrating that our proposed method significantly enhances self-awareness of the model in these tasks and reduces hallucinations.
  arXiv  Detail & Related papers  (2025-03-26T16:05:01Z)
• Creation-MMBench: Assessing Context-Aware Creative Intelligence in MLLM [58.42678619252968]
  Creation-MMBench is a benchmark designed to evaluate the creative capabilities of Multimodal Large Language Models. The benchmark comprises 765 test cases spanning 51 fine-grained tasks. Experimental results reveal that open-source MLLMs significantly underperform compared to proprietary models in creative tasks.
  arXiv  Detail & Related papers  (2025-03-18T17:51:34Z)
• A Causality-aware Paradigm for Evaluating Creativity of Multimodal Large Language Models [100.16387798660833]
  Oogiri game is a creativity-driven task requiring humor and associative thinking.<n>LoTbench is an interactive, causality-aware evaluation framework.<n>Results show that while most LLMs exhibit constrained creativity, the performance gap between LLMs and humans is not insurmountable.
  arXiv  Detail & Related papers  (2025-01-25T09:11:15Z)
• CAP: Evaluation of Persuasive and Creative Image Generation [28.49695567630899]
  We introduce three evaluation metrics to assess Creativity, prompt Alignment, and Persuasiveness in generated advertisement images. Our findings reveal that current Text-to-Image models struggle with creativity, persuasiveness, and alignment when the input text is implicit messages. We introduce a simple yet effective approach to enhance T2I models' capabilities in producing images that are better aligned, more creative, and more persuasive.
  arXiv  Detail & Related papers  (2024-12-10T19:54:59Z)
• Steering Large Language Models to Evaluate and Amplify Creativity [7.031631627161492]
  We show that we can leverage this knowledge of how to write creatively in order to better judge what is creative.<n>We take a mechanistic approach that extracts differences in the internal states of an LLM when prompted to respond "boringly" or "creatively"
  arXiv  Detail & Related papers  (2024-12-08T20:28:48Z)
• KITTEN: A Knowledge-Intensive Evaluation of Image Generation on Visual Entities [93.74881034001312]
  We conduct a systematic study on the fidelity of entities in text-to-image generation models. We focus on their ability to generate a wide range of real-world visual entities, such as landmark buildings, aircraft, plants, and animals. Our findings reveal that even the most advanced text-to-image models often fail to generate entities with accurate visual details.
  arXiv  Detail & Related papers  (2024-10-15T17:50:37Z)
• Aligning Vision Models with Human Aesthetics in Retrieval: Benchmarks and Algorithms [91.19304518033144]
  We aim to align vision models with human aesthetic standards in a retrieval system. We propose a preference-based reinforcement learning method that fine-tunes the vision models to better align the vision models with human aesthetics.
  arXiv  Detail & Related papers  (2024-06-13T17:59:20Z)
• Can AI Be as Creative as Humans? [84.43873277557852]
  We prove in theory that AI can be as creative as humans under the condition that it can properly fit the data generated by human creators. The debate on AI's creativity is reduced into the question of its ability to fit a sufficient amount of data.
  arXiv  Detail & Related papers  (2024-01-03T08:49:12Z)
• DeepCreativity: Measuring Creativity with Deep Learning Techniques [2.5426469613007012]
  This paper explores the possibility of using generative learning techniques for automatic assessment of creativity. We introduce a new measure, namely DeepCreativity, based on Margaret Boden's definition of creativity as composed by value, novelty and surprise.
  arXiv  Detail & Related papers  (2022-01-16T19:00:01Z)
• Creativity of Deep Learning: Conceptualization and Assessment [1.5738019181349994]
  We use insights from computational creativity to conceptualize and assess current applications of generative deep learning in creative domains. We highlight parallels between current systems and different models of human creativity as well as their shortcomings.
  arXiv  Detail & Related papers  (2020-12-03T21:44:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.