StoryAlign: Evaluating and Training Reward Models for Story Generation

• URL: http://arxiv.org/abs/2605.04831v1
• Date: Wed, 06 May 2026 12:28:17 GMT
• Title: StoryAlign: Evaluating and Training Reward Models for Story Generation
• Abstract summary: We introduce StoryRMB, the first benchmark for assessing reward models on story preferences.<n>We develop StoryReward, an advanced reward model for story preference trained on this dataset.<n>StoryReward achieves state-of-the-art (SoTA) performance on StoryRMB, outperforming much larger models.
• Score: 52.44891270363296
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Story generation aims to automatically produce coherent, structured, and engaging narratives. Although large language models (LLMs) have significantly advanced text generation, stories generated by LLMs still diverge from human-authored works regarding complex narrative structure and human-aligned preferences. A key reason is the absence of effective modeling of human story preferences, which are inherently subjective and under-explored. In this work, we systematically evaluate the modeling of human story preferences and introduce StoryRMB, the first benchmark for assessing reward models on story preferences. StoryRMB contains $1,133$ high-quality, human-verified instances, each consisting of a prompt, one chosen story, and three rejected stories. We find existing reward models struggle to select human-preferred stories, with the best model achieving only $66.3\%$ accuracy. To address this limitation, we construct roughly $100,000$ high-quality story preference pairs across diverse domains and develop StoryReward, an advanced reward model for story preference trained on this dataset. StoryReward achieves state-of-the-art (SoTA) performance on StoryRMB, outperforming much larger models. We also adopt StoryReward in downstream test-time scaling applications for best-of-n (BoN) story selection and find that it generally chooses stories better aligned with human preferences. We will release our dataset, model, and code to facilitate future research. Related code and data are available at https://github.com/THU-KEG/StoryReward.

Related papers

• StoryWriter: A Multi-Agent Framework for Long Story Generation [53.80343104003837]
  Long story generation remains a challenge for existing large language models.<n>We propose StoryWriter, a multi-agent story generation framework, which consists of three main modules.<n>StoryWriter significantly outperforms existing story generation baselines in both story quality and length.
  arXiv  Detail & Related papers  (2025-06-19T16:26:58Z)
• Learning to Reason for Long-Form Story Generation [84.09733333295338]
  We propose a general story-generation task (Next-Chapter Prediction) and a reward formulation (Verified Rewards via Completion Likelihood Improvement)<n>We learn to reason over a story's condensed information and generate a detailed plan for the next chapter.<n>Our reasoning is evaluated via the chapters it helps a story-generator create, and compared against non-trained and supervised finetuning (SFT) baselines.
  arXiv  Detail & Related papers  (2025-03-28T18:48:26Z)
• Generating Visual Stories with Grounded and Coreferent Characters [63.07511918366848]
  We present the first model capable of predicting visual stories with consistently grounded and coreferent character mentions.<n>Our model is finetuned on a new dataset which we build on top of the widely used VIST benchmark.<n>We also propose new evaluation metrics to measure the richness of characters and coreference in stories.
  arXiv  Detail & Related papers  (2024-09-20T14:56:33Z)
• TARN-VIST: Topic Aware Reinforcement Network for Visual Storytelling [14.15543866199545]
  As a cross-modal task, visual storytelling aims to generate a story for an ordered image sequence automatically. We propose a novel method, Topic Aware Reinforcement Network for VIsual StoryTelling (TARN-VIST) In particular, we pre-extracted the topic information of stories from both visual and linguistic perspectives.
  arXiv  Detail & Related papers  (2024-03-18T08:01:23Z)
• The Next Chapter: A Study of Large Language Models in Storytelling [51.338324023617034]
  The application of prompt-based learning with large language models (LLMs) has exhibited remarkable performance in diverse natural language processing (NLP) tasks. This paper conducts a comprehensive investigation, utilizing both automatic and human evaluation, to compare the story generation capacity of LLMs with recent models. The results demonstrate that LLMs generate stories of significantly higher quality compared to other story generation models.
  arXiv  Detail & Related papers  (2023-01-24T02:44:02Z)
• Robust Preference Learning for Storytelling via Contrastive Reinforcement Learning [53.92465205531759]
  Controlled automated story generation seeks to generate natural language stories satisfying constraints from natural language critiques or preferences. We train a contrastive bi-encoder model to align stories with human critiques, building a general purpose preference model. We further fine-tune the contrastive reward model using a prompt-learning technique to increase story generation robustness.
  arXiv  Detail & Related papers  (2022-10-14T13:21:33Z)
• Guiding Neural Story Generation with Reader Models [5.935317028008691]
  We introduce Story generation with Reader Models (StoRM), a framework in which a reader model is used to reason about the story should progress. Experiments show that our model produces significantly more coherent and on-topic stories, outperforming baselines in dimensions including plot plausibility and staying on topic.
  arXiv  Detail & Related papers  (2021-12-16T03:44:01Z)
• STORIUM: A Dataset and Evaluation Platform for Machine-in-the-Loop Story Generation [48.56586847883825]
  We introduce a dataset and evaluation platform built from STORIUM, an online collaborative storytelling community. Our dataset contains 6K lengthy stories with fine-grained natural language annotations interspersed throughout each narrative. We evaluate language models fine-tuned on our dataset by integrating them onto STORIUM, where real authors can query a model for suggested story continuations and then edit them.
  arXiv  Detail & Related papers  (2020-10-04T23:26:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.