Spoiler Alert: Narrative Forecasting as a Metric for Tension in LLM Storytelling

• URL: http://arxiv.org/abs/2604.09854v1
• Date: Fri, 10 Apr 2026 19:33:43 GMT
• Title: Spoiler Alert: Narrative Forecasting as a Metric for Tension in LLM Storytelling
• Abstract summary: We argue that existing rubrics overlook a key dimension of compelling human stories: narrative tension.<n>We introduce the 100-Endings metric, which walks through a story sentence by sentence.<n>Unlike rubric-based judges, 100-Endings correctly ranks New Yorker stories far above LLM outputs.
• Score: 15.25806708314033
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLMs have so far failed both to generate consistently compelling stories and to recognize this failure--on the leading creative-writing benchmark (EQ-Bench), LLM judges rank zero-shot AI stories above New Yorker short stories, a gold standard for literary fiction. We argue that existing rubrics overlook a key dimension of compelling human stories: narrative tension. We introduce the 100-Endings metric, which walks through a story sentence by sentence: at each position, a model predicts how the story will end 100 times given only the text so far, and we measure tension as how often predictions fail to match the ground truth. Beyond the mismatch rate, the sentence-level curve yields complementary statistics, such as inflection rate, a geometric measure of how frequently the curve reverses direction, tracking twists and revelations. Unlike rubric-based judges, 100-Endings correctly ranks New Yorker stories far above LLM outputs. Grounded in narratological principles, we design a story-generation pipeline using structural constraints, including analysis of story templates, idea formulation, and narrative scaffolding. Our pipeline significantly increases narrative tension as measured by the 100-Endings metric, while maintaining performance on the EQ-Bench leaderboard.

Related papers

• From Personas to Plot: Character-Grounded Multi-Agent Story Generation for Long-Form Narratives [3.0130431031064613]
  We introduce a unified framework for long-form narrative generation and verification.<n>We show that our framework produces coherent narratives compared to single-model prompting and IBSEN.<n>These results suggest that long-form narratives can emerge from explicit world-state tracking and goal-driven multi-agent generation.
  arXiv  Detail & Related papers  (2026-07-01T13:23:07Z)
• NARRA-Gym for Evaluating Interactive Narrative Agents [69.49891044929372]
  We introduce NARRA-Gym, an evaluation environment that turns a sparse emotional seed into a complete interactive story episode.<n>We evaluate nine frontier LLMs using a controlled LLM-as-judge sweep over eight benchmark personas and a human evaluation in which participants rate customized model outputs.
  arXiv  Detail & Related papers  (2026-05-08T21:36:23Z)
• Lost in Stories: Consistency Bugs in Long Story Generation by LLMs [21.241285394393632]
  ConStory-Bench is a benchmark designed to evaluate narrative consistency in long-form story generation.<n>It contains 2,000 prompts across four task scenarios and defines a taxonomy of five error categories with 19 fine-grained subtypes.<n>We also develop ConStory-Checker, an automated pipeline that detects contradictions and grounds each judgment in explicit textual evidence.
  arXiv  Detail & Related papers  (2026-03-06T04:16:46Z)
• Codified Foreshadowing-Payoff Text Generation [67.01182739162142]
  Foreshadowing and payoff are ubiquitous narrative devices through which authors introduce commitments early in a story and resolve them through concrete, observable outcomes.<n>Existing evaluations largely overlook this structural failure, focusing on surface-level coherence rather than the logical fulfillment of narrative setups.<n>We introduce Codified Foreshadowing-Payoff Generation, a novel framework that reframes narrative quality through the lens of payoff realization.
  arXiv  Detail & Related papers  (2026-01-11T19:05:37Z)
• Living the Novel: A System for Generating Self-Training Timeline-Aware Conversational Agents from Novels [50.43968216132018]
  We present an end-to-end system that transforms any literary work into an immersive, multi-character conversational experience.<n>This system is designed to solve two fundamental challenges for LLM-driven characters.
  arXiv  Detail & Related papers  (2025-12-08T11:57:46Z)
• NOAH: Benchmarking Narrative Prior driven Hallucination and Omission in Video Large Language Models [8.6767620170781]
  Video large language models (Video LLMs) have recently achieved strong performance on tasks such as captioning, summarization, and question answering.<n>Many models and training methods explicitly encourage continuity across events to enhance narrative coherence.<n>We identify this bias, which we call narrative prior, as a key driver of two errors: hallucinations, where non-existent events are introduced or existing ones are misinterpreted, and omissions, where factual events are suppressed because they are misaligned with surrounding context.
  arXiv  Detail & Related papers  (2025-11-09T17:41:11Z)
• Modeling Fair Play in Detective Stories with Language Models [14.90042250991032]
  We present a probabilistic framework for detective fiction that allows us to define desired qualities.<n>We validate the framework by applying it to LLM-generated detective stories.<n>Results show that while LLM-generated stories may be unpredictable, they generally fail to balance the trade-off between surprise and fair play.
  arXiv  Detail & Related papers  (2025-07-18T11:55:18Z)
• Failure Modes of LLMs for Causal Reasoning on Narratives [51.19592551510628]
  We investigate the interaction between world knowledge and logical reasoning.<n>We find that state-of-the-art large language models (LLMs) often rely on superficial generalizations.<n>We show that simple reformulations of the task can elicit more robust reasoning behavior.
  arXiv  Detail & Related papers  (2024-10-31T12:48:58Z)
• Generating Visual Stories with Grounded and Coreferent Characters [63.07511918366848]
  We present the first model capable of predicting visual stories with consistently grounded and coreferent character mentions.<n>Our model is finetuned on a new dataset which we build on top of the widely used VIST benchmark.<n>We also propose new evaluation metrics to measure the richness of characters and coreference in stories.
  arXiv  Detail & Related papers  (2024-09-20T14:56:33Z)
• Are Large Language Models Capable of Generating Human-Level Narratives? [114.34140090869175]
  This paper investigates the capability of LLMs in storytelling, focusing on narrative development and plot progression. We introduce a novel computational framework to analyze narratives through three discourse-level aspects. We show that explicit integration of discourse features can enhance storytelling, as is demonstrated by over 40% improvement in neural storytelling.
  arXiv  Detail & Related papers  (2024-07-18T08:02:49Z)
• LongStory: Coherent, Complete and Length Controlled Long story Generation [16.804434185847363]
  We present the LongStory for coherent, complete, and length-controlled long story generation.<n>LongStory introduces two novel methodologies: (1) the long and short-term contexts weight calibrator (CWC) and (2) long story structural positions (LSP).
  arXiv  Detail & Related papers  (2023-11-26T06:24:25Z)
• Paragraph-level Commonsense Transformers with Recurrent Memory [77.4133779538797]
  We train a discourse-aware model that incorporates paragraph-level information to generate coherent commonsense inferences from narratives. Our results show that PARA-COMET outperforms the sentence-level baselines, particularly in generating inferences that are both coherent and novel.
  arXiv  Detail & Related papers  (2020-10-04T05:24:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.