CreativeBench: Benchmarking and Enhancing Machine Creativity via Self-Evolving Challenges

• URL: http://arxiv.org/abs/2603.11863v1
• Date: Thu, 12 Mar 2026 12:36:56 GMT
• Title: CreativeBench: Benchmarking and Enhancing Machine Creativity via Self-Evolving Challenges
• Abstract summary: We introduce CreativeBench, a benchmark for evaluating machine creativity in code generation.<n>CreativeBench objectively distinguishes creativity from hallucination via a unified metric defined as the product of quality and novelty.<n>We propose EvoRePE, a plug-and-play inference-time steering strategy that internalizes evolutionary search patterns to consistently enhance machine creativity.
• Score: 69.3795501613098
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The saturation of high-quality pre-training data has shifted research focus toward evolutionary systems capable of continuously generating novel artifacts, leading to the success of AlphaEvolve. However, the progress of such systems is hindered by the lack of rigorous, quantitative evaluation. To tackle this challenge, we introduce CreativeBench, a benchmark for evaluating machine creativity in code generation, grounded in a classical cognitive framework. Comprising two subsets -- CreativeBench-Combo and CreativeBench-Explore -- the benchmark targets combinatorial and exploratory creativity through an automated pipeline utilizing reverse engineering and self-play. By leveraging executable code, CreativeBench objectively distinguishes creativity from hallucination via a unified metric defined as the product of quality and novelty. Our analysis of state-of-the-art models reveals distinct behaviors: (1) scaling significantly improves combinatorial creativity but yields diminishing returns for exploration; (2) larger models exhibit ``convergence-by-scaling,'' becoming more correct but less divergent; and (3) reasoning capabilities primarily benefit constrained exploration rather than combination. Finally, we propose EvoRePE, a plug-and-play inference-time steering strategy that internalizes evolutionary search patterns to consistently enhance machine creativity.

Related papers

• Measuring Creativity in the Age of Generative AI: Distinguishing Human and AI-Generated Creative Performance in Hiring and Talent Systems [0.0]
  This paper reconceptualizes creativity as a distributional and process-based property that emerges under shared constraints and competitive incentives.<n>We introduce a quantitative framework for measuring creativity as novelty in synthesis, operationalized through idea generation and idea transformation within embedding space.<n>The findings suggest that in the age of generative AI, distinctiveness rather than fluency becomes the primary signal of human creative capability.
  arXiv  Detail & Related papers  (2026-04-10T12:49:17Z)
• CreativityBench: Evaluating Agent Creative Reasoning via Affordance-Based Tool Repurposing [67.2958108309687]
  CreativityBench is a benchmark for evaluating affordance-based creativity in LLMs.<n>We generate 14K grounded tasks that require identifying non-obvious yet physically plausible solutions under constraints.<n> Evaluations across 10 state-of-the-art LLMs, including closed and open-source models, show that models can often select a plausible object, but fail to identify the correct parts.
  arXiv  Detail & Related papers  (2026-04-06T16:33:23Z)
• CREATE: Testing LLMs for Associative Creativity [51.64544603712446]
  We introduce CREATE, a benchmark designed to evaluate models' capacity for creative associative reasoning.<n>We show that the strongest models achieve higher creative utility than others.<n>Our results illustrate that thinking models are not always more effective on our task, even with high token budgets.
  arXiv  Detail & Related papers  (2026-03-10T17:58:44Z)
• What Shapes a Creative Machine Mind? Comprehensively Benchmarking Creativity in Foundation Models [16.81217474424392]
  We introduce C2-Eval, a holistic benchmark for unified assessment of creativity in foundation models (FMs)<n>C2-Eval distinguishes between two complementary forms of creativity: convergent creativity, where tasks admit constrained solutions, and divergent creativity, where tasks are open-ended.<n>Our results show that C2-Eval is an effective lens for examining the evolving landscape of creative AI.
  arXiv  Detail & Related papers  (2025-10-05T03:00:50Z)
• Probing and Inducing Combinational Creativity in Vision-Language Models [52.76981145923602]
  Recent advances in Vision-Language Models (VLMs) have sparked debate about whether their outputs reflect combinational creativity.<n>We propose the Identification-Explanation-Implication (IEI) framework, which decomposes creative processes into three levels.<n>To validate this framework, we curate CreativeMashup, a high-quality dataset of 666 artist-generated visual mashups annotated according to the IEI framework.
  arXiv  Detail & Related papers  (2025-04-17T17:38:18Z)
• Automating Creativity [1.0200170217746136]
  This paper explores what is required to evolve AI from generative to creative. We develop a triple prompt-response-reward engineering framework to develop the creative capability of GenAI.
  arXiv  Detail & Related papers  (2024-05-11T05:05:10Z)
• Can AI Be as Creative as Humans? [84.43873277557852]
  We prove in theory that AI can be as creative as humans under the condition that it can properly fit the data generated by human creators. The debate on AI's creativity is reduced into the question of its ability to fit a sufficient amount of data.
  arXiv  Detail & Related papers  (2024-01-03T08:49:12Z)
• Towards Creativity Characterization of Generative Models via Group-based Subset Scanning [64.6217849133164]
  We propose group-based subset scanning to identify, quantify, and characterize creative processes. We find that creative samples generate larger subsets of anomalies than normal or non-creative samples across datasets.
  arXiv  Detail & Related papers  (2022-03-01T15:07:14Z)
• Towards creativity characterization of generative models via group-based subset scanning [51.84144826134919]
  We propose group-based subset scanning to quantify, detect, and characterize creative processes. Creative samples generate larger subsets of anomalies than normal or non-creative samples across datasets.
  arXiv  Detail & Related papers  (2021-04-01T14:07:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.