Creative Preference Optimization

• URL: http://arxiv.org/abs/2505.14442v1
• Date: Tue, 20 May 2025 14:43:41 GMT
• Title: Creative Preference Optimization
• Authors: Mete Ismayilzada, Antonio Laverghetta Jr., Simone A. Luchini, Reet Patel, Antoine Bosselut, Lonneke van der Plas, Roger Beaty,
• Abstract summary: We propose a novel alignment method that injects signals from multiple creativity dimensions into the preference optimization objective in a modular fashion.<n>We train and evaluate creativity-augmented versions of several models using CrPO and MuCE, a new large-scale human preference dataset.<n>Our models outperform strong baselines, including GPT-4o, on both automated and human evaluations.
• Score: 13.266256662558822
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While Large Language Models (LLMs) have demonstrated impressive performance across natural language generation tasks, their ability to generate truly creative content-characterized by novelty, diversity, surprise, and quality-remains limited. Existing methods for enhancing LLM creativity often focus narrowly on diversity or specific tasks, failing to address creativity's multifaceted nature in a generalizable way. In this work, we propose Creative Preference Optimization (CrPO), a novel alignment method that injects signals from multiple creativity dimensions into the preference optimization objective in a modular fashion. We train and evaluate creativity-augmented versions of several models using CrPO and MuCE, a new large-scale human preference dataset spanning over 200,000 human-generated responses and ratings from more than 30 psychological creativity assessments. Our models outperform strong baselines, including GPT-4o, on both automated and human evaluations, producing more novel, diverse, and surprising generations while maintaining high output quality. Additional evaluations on NoveltyBench further confirm the generalizability of our approach. Together, our results demonstrate that directly optimizing for creativity within preference frameworks is a promising direction for advancing the creative capabilities of LLMs without compromising output quality.

Related papers

• Rethinking Creativity Evaluation: A Critical Analysis of Existing Creativity Evaluations [47.51585845370428]
  We examine creativity measures including the creativity index, perplexity, syntactic templates, and LLM-as-a-Judge.<n>Our analyses reveal that these metrics exhibit limited consistency, capturing different dimensions of creativity.
  arXiv  Detail & Related papers  (2025-08-07T15:11:48Z)
• Cooking Up Creativity: A Cognitively-Inspired Approach for Enhancing LLM Creativity through Structured Representations [53.950760059792614]
  Large Language Models (LLMs) excel at countless tasks, yet struggle with creativity.<n>We introduce a novel approach that couples LLMs with structured representations and cognitively inspired manipulations to generate more creative and diverse ideas.<n>We demonstrate our approach in the culinary domain with DishCOVER, a model that generates creative recipes.
  arXiv  Detail & Related papers  (2025-04-29T11:13:06Z)
• Probing and Inducing Combinational Creativity in Vision-Language Models [52.76981145923602]
  Recent advances in Vision-Language Models (VLMs) have sparked debate about whether their outputs reflect combinational creativity.<n>We propose the Identification-Explanation-Implication (IEI) framework, which decomposes creative processes into three levels.<n>To validate this framework, we curate CreativeMashup, a high-quality dataset of 666 artist-generated visual mashups annotated according to the IEI framework.
  arXiv  Detail & Related papers  (2025-04-17T17:38:18Z)
• Evaluating the Diversity and Quality of LLM Generated Content [72.84945252821908]
  We introduce a framework for measuring effective semantic diversity--diversity among outputs that meet quality thresholds.<n>Although preference-tuned models exhibit reduced lexical and syntactic diversity, they produce greater effective semantic diversity than SFT or base models.<n>These findings have important implications for applications that require diverse yet high-quality outputs.
  arXiv  Detail & Related papers  (2025-04-16T23:02:23Z)
• Creation-MMBench: Assessing Context-Aware Creative Intelligence in MLLM [58.42678619252968]
  Creation-MMBench is a benchmark designed to evaluate the creative capabilities of Multimodal Large Language Models.<n>The benchmark comprises 765 test cases spanning 51 fine-grained tasks.<n> Experimental results reveal that open-source MLLMs significantly underperform compared to proprietary models in creative tasks.
  arXiv  Detail & Related papers  (2025-03-18T17:51:34Z)
• A Causality-aware Paradigm for Evaluating Creativity of Multimodal Large Language Models [100.16387798660833]
  Oogiri game is a creativity-driven task requiring humor and associative thinking.<n>LoTbench is an interactive, causality-aware evaluation framework.<n>Results show that while most LLMs exhibit constrained creativity, the performance gap between LLMs and humans is not insurmountable.
  arXiv  Detail & Related papers  (2025-01-25T09:11:15Z)
• The creative psychometric item generator: a framework for item generation and validation using large language models [1.765099515298011]
  Large language models (LLMs) are being used to automate workplace processes requiring a high degree of creativity. We develop a psychometrically inspired framework for creating test items for a classic free-response creativity test: the creative problem-solving (CPS) task. We find strong empirical evidence that CPIG generates valid and reliable items and that this effect is not attributable to known biases in the evaluation process.
  arXiv  Detail & Related papers  (2024-08-30T18:31:02Z)
• Benchmarking Language Model Creativity: A Case Study on Code Generation [39.546827184857754]
  In this work, we introduce a framework for quantifying LLM creativity.<n>We define NEOGAUGE, a metric that quantifies both convergent and divergent thinking in the generated creative responses.<n>We test the proposed framework on Codeforces problems, which serve as both a natural dataset for coding tasks and a collection of prior human solutions.
  arXiv  Detail & Related papers  (2024-07-12T05:55:22Z)
• Divergent Creativity in Humans and Large Language Models [37.67363469600804]
  The recent surge in the capabilities of Large Language Models has led to claims that they are approaching a level of creativity akin to human capabilities. We leverage recent advances in creativity science to build a framework for in-depth analysis of divergent creativity in both state-of-the-art LLMs and a substantial dataset of 100,000 humans.
  arXiv  Detail & Related papers  (2024-05-13T22:37:52Z)
• Assessing and Understanding Creativity in Large Language Models [33.37237667182931]
  This paper aims to establish an efficient framework for assessing the level of creativity in large language models (LLMs) By adapting the Torrance Tests of Creative Thinking, the research evaluates the creative performance of various LLMs across 7 tasks. We found that the creativity of LLMs primarily falls short in originality, while excelling in elaboration.
  arXiv  Detail & Related papers  (2024-01-23T05:19:47Z)
• A New Creative Generation Pipeline for Click-Through Rate with Stable Diffusion Model [8.945197427679924]
  Traditional AI-based approaches face the same problem of not considering user information while having limited aesthetic knowledge from designers. To optimize the results, the generated creatives in traditional methods are then ranked by another module named creative ranking model. This paper proposes a new automated Creative Generation pipeline for Click-Through Rate (CG4CTR) with the goal of improving CTR during the creative generation stage.
  arXiv  Detail & Related papers  (2024-01-17T03:27:39Z)
• Can AI Be as Creative as Humans? [84.43873277557852]
  We prove in theory that AI can be as creative as humans under the condition that it can properly fit the data generated by human creators. The debate on AI's creativity is reduced into the question of its ability to fit a sufficient amount of data.
  arXiv  Detail & Related papers  (2024-01-03T08:49:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.