CresOWLve: Benchmarking Creative Problem-Solving Over Real-World Knowledge

• URL: http://arxiv.org/abs/2604.03374v1
• Date: Fri, 03 Apr 2026 18:08:14 GMT
• Title: CresOWLve: Benchmarking Creative Problem-Solving Over Real-World Knowledge
• Abstract summary: We introduce CresOWLve, a benchmark for evaluating creative problem-solving using puzzles grounded in real-world knowledge.<n> Problems in CresOWLve require employing multiple creative thinking strategies, retrieving facts from diverse domains, and creatively combining them to arrive at a solution.<n>Our analysis reveals a consistent performance gap: models perform substantially better on factual questions than on creative ones.
• Score: 19.526111468269892
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Creative problem-solving requires combining multiple cognitive abilities, including logical reasoning, lateral thinking, analogy-making, and commonsense knowledge, to discover insights that connect seemingly unrelated pieces of information. However, most existing benchmarks for large language models (LLMs) evaluate only specific components of this process. Moreover, many creativity-oriented benchmarks rely on artificially constructed brainteasers or contrived scenarios that do not reflect how creative problem-solving occurs in real-world settings. To address this gap, we introduce CresOWLve, a benchmark for evaluating creative problem-solving using puzzles grounded in real-world knowledge. Problems in CresOWLve require employing multiple creative thinking strategies, retrieving facts from diverse domains, and creatively combining them to arrive at a solution. Evaluating several frontier non-thinking and thinking LLMs, we show that CresOWLve remains highly challenging. Our analysis reveals a consistent performance gap: models perform substantially better on factual questions than on creative ones (up to a -17% drop). While models can often retrieve the relevant knowledge, they struggle to form the non-obvious creative connections required to integrate this information and arrive at the correct answer.

Related papers

• CREATE: Testing LLMs for Associative Creativity [51.64544603712446]
  We introduce CREATE, a benchmark designed to evaluate models' capacity for creative associative reasoning.<n>We show that the strongest models achieve higher creative utility than others.<n>Our results illustrate that thinking models are not always more effective on our task, even with high token budgets.
  arXiv  Detail & Related papers  (2026-03-10T17:58:44Z)
• Divergent-Convergent Thinking in Large Language Models for Creative Problem Generation [22.659182231103443]
  Large language models (LLMs) can generate educational questions and problems, enabling educators to create large-scale learning materials.<n>LLMs are limited by the Artificial Hivemind'' effect, where they generate similar responses within the same model and produce homogeneous outputs across different models.<n>We propose CreativeDC, a two-phase prompting method that explicitly scaffolds the LLM's reasoning into distinct phases.
  arXiv  Detail & Related papers  (2025-12-29T16:53:48Z)
• Universe of Thoughts: Enabling Creative Reasoning with Large Language Models [1.9480051045857554]
  We introduce a computational framework for creative reasoning inspired by cognitive science principles.<n>We propose three core creative reasoning paradigms, namely, textitexploratory, textittransformative, and textitUoT, for short.<n>We show that UoT demonstrates superior performance in creative reasoning.
  arXiv  Detail & Related papers  (2025-11-25T16:34:59Z)
• CreativityPrism: A Holistic Benchmark for Large Language Model Creativity [64.18257552903151]
  Creativity is often seen as a hallmark of human intelligence.<n>There is still no holistic framework to evaluate their creativity across diverse scenarios.<n>We propose CreativityPrism, an evaluation analysis framework that decomposes creativity into three dimensions: quality, novelty, and diversity.
  arXiv  Detail & Related papers  (2025-10-23T00:22:10Z)
• Creation-MMBench: Assessing Context-Aware Creative Intelligence in MLLM [58.42678619252968]
  Creation-MMBench is a benchmark designed to evaluate the creative capabilities of Multimodal Large Language Models.<n>The benchmark comprises 765 test cases spanning 51 fine-grained tasks.<n> Experimental results reveal that open-source MLLMs significantly underperform compared to proprietary models in creative tasks.
  arXiv  Detail & Related papers  (2025-03-18T17:51:34Z)
• A Causality-aware Paradigm for Evaluating Creativity of Multimodal Large Language Models [100.16387798660833]
  Oogiri game is a creativity-driven task requiring humor and associative thinking.<n>LoTbench is an interactive, causality-aware evaluation framework.<n>Results show that while most LLMs exhibit constrained creativity, the performance gap between LLMs and humans is not insurmountable.
  arXiv  Detail & Related papers  (2025-01-25T09:11:15Z)
• Benchmarking Language Model Creativity: A Case Study on Code Generation [39.546827184857754]
  In this work, we introduce a framework for quantifying LLM creativity.<n>We define NEOGAUGE, a metric that quantifies both convergent and divergent thinking in the generated creative responses.<n>We test the proposed framework on Codeforces problems, which serve as both a natural dataset for coding tasks and a collection of prior human solutions.
  arXiv  Detail & Related papers  (2024-07-12T05:55:22Z)
• SOK-Bench: A Situated Video Reasoning Benchmark with Aligned Open-World Knowledge [60.76719375410635]
  We propose a new benchmark (SOK-Bench) consisting of 44K questions and 10K situations with instance-level annotations depicted in the videos. The reasoning process is required to understand and apply situated knowledge and general knowledge for problem-solving. We generate associated question-answer pairs and reasoning processes, finally followed by manual reviews for quality assurance.
  arXiv  Detail & Related papers  (2024-05-15T21:55:31Z)
• MacGyver: Are Large Language Models Creative Problem Solvers? [87.70522322728581]
  We explore the creative problem-solving capabilities of modern LLMs in a novel constrained setting.<n>We create MACGYVER, an automatically generated dataset consisting of over 1,600 real-world problems.<n>We present our collection to both LLMs and humans to compare and contrast their problem-solving abilities.
  arXiv  Detail & Related papers  (2023-11-16T08:52:27Z)
• On the Creativity of Large Language Models [2.4555276449137042]
  Large Language Models (LLMs) are revolutionizing several areas of Artificial Intelligence.<n>This article first analyzes the development of LLMs under the lens of creativity theories.<n>Then, we consider different classic perspectives, namely product, process, press, and person.<n>Finally, we examine the societal impact of these technologies with a particular focus on the creative industries.
  arXiv  Detail & Related papers  (2023-03-27T18:00:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.