Playing with Words, Improving with Rewards: Training Language Models for Creative Association

• URL: http://arxiv.org/abs/2605.27832v1
• Date: Wed, 27 May 2026 01:41:22 GMT
• Title: Playing with Words, Improving with Rewards: Training Language Models for Creative Association
• Abstract summary: Large Language Models (LLMs) are being applied to increasingly difficult problems and use cases.<n>To navigate their vast solution spaces effectively, LLMs need to be creative.<n>We train LLMs on Codenames, a word-association game that exercises the two central axes of creativity.
• Score: 9.232251624834644
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) are being applied to increasingly difficult problems and use cases. To navigate their vast solution spaces effectively, LLMs need to be creative. Yet the subjective nature of creativity and the limits of human judgment make training LLMs for creativity especially challenging. As a solution, we train LLMs on Codenames, a word-association game that exercises the two central axes of creativity, divergent and convergent thinking, while yielding objectively verifiable outcomes. This verifiability lets us bypass human judgment and train with Reinforcement Learning with Verifiable Rewards (RLVR). We train Qwen3-1.7B, 4B, and 8B models and evaluate them on ten creativity and four reasoning benchmarks. We find that the precision-diversity trade-off is scale-dependent: the 8B model prioritizes creativity over precision, while the 1.7B and 4B models gain reasoning precision at the cost of creativity. Concretely, the 8B model shows modest but consistent creativity gains (8 of 10 benchmarks) with only minor reasoning degradation, whereas the smaller models achieve substantial gains on reasoning tasks. Our study presents a scalable and effective solution to train LLMs for creativity.

Related papers

• CreativeInstruct: Scalably Teaching LLMs to Balance Quality, Creativity, and Diversity [67.74087963315213]
  CreativeInstruct is an instruction-tuning method that teaches large language models to balance creative, base-model-like generations with the quality of post-trained models.<n>We show that CreativeInstruct matches or exceeds the diversity of both multi-model baselines and distilled variants of their outputs.<n>We also show the benefits of creative models as a substrate for RL: GRPO applied to a CreativeInstruct checkpoint improves by 4% on AMC and 5% points on MATH.
  arXiv  Detail & Related papers  (2026-08-07T17:55:48Z)
• CreativityBench: Evaluating Agent Creative Reasoning via Affordance-Based Tool Repurposing [67.2958108309687]
  CreativityBench is a benchmark for evaluating affordance-based creativity in LLMs.<n>We generate 14K grounded tasks that require identifying non-obvious yet physically plausible solutions under constraints.<n> Evaluations across 10 state-of-the-art LLMs, including closed and open-source models, show that models can often select a plausible object, but fail to identify the correct parts.
  arXiv  Detail & Related papers  (2026-04-06T16:33:23Z)
• Large Language Models Align with the Human Brain during Creative Thinking [19.08997828415446]
  We study brain alignment during creative thinking using fMRI data from 170 participants performing the Alternate Uses Task (AUT)<n>We find that brain-LLM alignment scales with model size (default mode network only) and idea originality (both networks)<n>We further show that post-training objectives shape alignment in functionally selective ways.
  arXiv  Detail & Related papers  (2026-04-03T22:02:15Z)
• CREATE: Testing LLMs for Associative Creativity [51.64544603712446]
  We introduce CREATE, a benchmark designed to evaluate models' capacity for creative associative reasoning.<n>We show that the strongest models achieve higher creative utility than others.<n>Our results illustrate that thinking models are not always more effective on our task, even with high token budgets.
  arXiv  Detail & Related papers  (2026-03-10T17:58:44Z)
• Beyond Divergent Creativity: A Human-Based Evaluation of Creativity in Large Language Models [6.036586911740041]
  Large language models (LLMs) are increasingly used in verbal creative tasks.<n>The widely used Divergent Association Task ( DAT) focuses on novelty, ignoring appropriateness.<n>We evaluate a range of state-of-the-art LLMs on DAT and show that their scores on the task are lower than those of two baselines that do not possess any creative abilities.
  arXiv  Detail & Related papers  (2026-01-28T12:41:32Z)
• A Causality-aware Paradigm for Evaluating Creativity of Multimodal Large Language Models [100.16387798660833]
  Oogiri game is a creativity-driven task requiring humor and associative thinking.<n>LoTbench is an interactive, causality-aware evaluation framework.<n>Results show that while most LLMs exhibit constrained creativity, the performance gap between LLMs and humans is not insurmountable.
  arXiv  Detail & Related papers  (2025-01-25T09:11:15Z)
• SuperCorrect: Advancing Small LLM Reasoning with Thought Template Distillation and Self-Correction [89.56181323849512]
  SuperCorrect is a novel two-stage framework that uses a large teacher model to supervise and correct both the reasoning and reflection processes of a smaller student model.<n>In the first stage, we extract hierarchical high-level and detailed thought templates from the teacher model to guide the student model in eliciting more fine-grained reasoning thoughts.<n>In the second stage, we introduce cross-model collaborative direct preference optimization (DPO) to enhance the self-correction abilities of the student model.
  arXiv  Detail & Related papers  (2024-10-11T17:25:52Z)
• Benchmarking Language Model Creativity: A Case Study on Code Generation [39.546827184857754]
  In this work, we introduce a framework for quantifying LLM creativity.<n>We define NEOGAUGE, a metric that quantifies both convergent and divergent thinking in the generated creative responses.<n>We test the proposed framework on Codeforces problems, which serve as both a natural dataset for coding tasks and a collection of prior human solutions.
  arXiv  Detail & Related papers  (2024-07-12T05:55:22Z)
• Creativity Has Left the Chat: The Price of Debiasing Language Models [1.223779595809275]
  We investigate the unintended consequences of Reinforcement Learning from Human Feedback on the creativity of Large Language Models (LLMs) Our findings have significant implications for marketers who rely on LLMs for creative tasks such as copywriting, ad creation, and customer persona generation.
  arXiv  Detail & Related papers  (2024-06-08T22:14:51Z)
• The ART of LLM Refinement: Ask, Refine, and Trust [85.75059530612882]
  We propose a reasoning with refinement objective called ART: Ask, Refine, and Trust. It asks necessary questions to decide when an LLM should refine its output. It achieves a performance gain of +5 points over self-refinement baselines.
  arXiv  Detail & Related papers  (2023-11-14T07:26:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.