Modelling Suspense in Short Stories as Uncertainty Reduction over Neural Representation

• URL: http://arxiv.org/abs/2004.14905v1
• Date: Thu, 30 Apr 2020 16:03:06 GMT
• Title: Modelling Suspense in Short Stories as Uncertainty Reduction over Neural Representation
• Abstract summary: Suspense is a crucial ingredient of narrative fiction, engaging readers and making stories compelling. We compare two ways for modelling suspense: surprise, a backward-looking measure of how unexpected the current state is given the story so far; and uncertainty reduction, a forward-looking measure of how unexpected the continuation of the story is. We propose a hierarchical language model that encodes stories and computes surprise and uncertainty reduction.
• Score: 17.683705809784453
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Suspense is a crucial ingredient of narrative fiction, engaging readers and making stories compelling. While there is a vast theoretical literature on suspense, it is computationally not well understood. We compare two ways for modelling suspense: surprise, a backward-looking measure of how unexpected the current state is given the story so far; and uncertainty reduction, a forward-looking measure of how unexpected the continuation of the story is. Both can be computed either directly over story representations or over their probability distributions. We propose a hierarchical language model that encodes stories and computes surprise and uncertainty reduction. Evaluating against short stories annotated with human suspense judgements, we find that uncertainty reduction over representations is the best predictor, resulting in near-human accuracy. We also show that uncertainty reduction can be used to predict suspenseful events in movie synopses.

Related papers

• Probing for Reading Times [60.30314214800421]
  We probe language model representations for human reading times in English, Greek, Hebrew, Russian, and Turkish.<n>We find that the representations from early layers outperform surprisal in predicting early-pass measures such as first fixation and gaze duration.
  arXiv  Detail & Related papers  (2026-04-20T18:12:59Z)
• Modeling Fair Play in Detective Stories with Language Models [14.90042250991032]
  We present a probabilistic framework for detective fiction that allows us to define desired qualities.<n>We validate the framework by applying it to LLM-generated detective stories.<n>Results show that while LLM-generated stories may be unpredictable, they generally fail to balance the trade-off between surprise and fair play.
  arXiv  Detail & Related papers  (2025-07-18T11:55:18Z)
• A Spatio-Temporal Point Process for Fine-Grained Modeling of Reading Behavior [47.47269936037604]
  Ansatz of psycholinguistics is that modeling a reader's fixations andcades yields insight into their online sentence processing.<n>Standard approaches to such modeling rely on eye-tracking measurements aggregated and models that impose strong assumptions.<n>In this paper, we propose a more general probabilistic model of reading behavior, based on a marked sac-temporal point process.<n>The saccades are modeled using a Hawkes process, which captures how each fixation excites the probability of a new fixation occurring near it in time and space.
  arXiv  Detail & Related papers  (2025-06-24T20:39:21Z)
• Counterfactual reasoning: an analysis of in-context emergence [57.118735341305786]
  We show that language models are capable of counterfactual reasoning.<n>We find that self-attention, model depth and pre-training data diversity drive performance.<n>Our findings extend to counterfactual reasoning under SDE dynamics.
  arXiv  Detail & Related papers  (2025-06-05T16:02:07Z)
• Measuring In-Context Computation Complexity via Hidden State Prediction [33.504027525492056]
  We show that a neural sequence model's ability to predict its own future hidden states correlates with the intuitive interestingness of the task. We propose a novel learned predictive prior that enables us to measure the novel information gained in each step. Our metric predicts the description length of formal languages learned in-context, the complexity of mathematical reasoning problems, and the correctness of self-generated reasoning chains.
  arXiv  Detail & Related papers  (2025-03-17T17:56:14Z)
• Enhancing Uncertainty Modeling with Semantic Graph for Hallucination Detection [46.930149191121416]
  Large Language Models (LLMs) are prone to hallucination with non-factual or unfaithful statements. We propose a method to enhance uncertainty modeling with semantic graph for hallucination detection.
  arXiv  Detail & Related papers  (2025-01-02T16:45:05Z)
• Creating Suspenseful Stories: Iterative Planning with Large Language Models [2.6923151107804055]
  We propose a novel iterative-prompting-based planning method that is grounded in two theoretical foundations of story suspense. To the best of our knowledge, this paper is the first attempt at suspenseful story generation with large language models.
  arXiv  Detail & Related papers  (2024-02-27T01:25:52Z)
• Predicting Emergent Abilities with Infinite Resolution Evaluation [85.89911520190711]
  We introduce PassUntil, an evaluation strategy with theoretically infinite resolution, through massive sampling in the decoding phase. We predict the performance of the 2.4B model on code generation with merely 0.05% deviation before training starts. We identify a kind of accelerated emergence whose scaling curve cannot be fitted by standard scaling law function.
  arXiv  Detail & Related papers  (2023-10-05T02:35:00Z)
• Neural Story Planning [8.600049807193413]
  We present an approach to story plot generation that unifies causal planning with neural language models. Our system infers the preconditions for events in the story and then events that will cause those conditions to become true. Results indicate that our proposed method produces more coherent plotlines than several strong baselines.
  arXiv  Detail & Related papers  (2022-12-16T21:29:41Z)
• Mutual Information Alleviates Hallucinations in Abstractive Summarization [73.48162198041884]
  We find a simple criterion under which models are significantly more likely to assign more probability to hallucinated content during generation: high model uncertainty. This finding offers a potential explanation for hallucinations: models default to favoring text with high marginal probability, when uncertain about a continuation. We propose a decoding strategy that switches to optimizing for pointwise mutual information of the source and target token--rather than purely the probability of the target token--when the model exhibits uncertainty.
  arXiv  Detail & Related papers  (2022-10-24T13:30:54Z)
• PROMPT WAYWARDNESS: The Curious Case of Discretized Interpretation of Continuous Prompts [99.03864962014431]
  Fine-tuning continuous prompts for target tasks has emerged as a compact alternative to full model fine-tuning. In practice, we observe a "wayward" behavior between the task solved by continuous prompts and their nearest neighbor.
  arXiv  Detail & Related papers  (2021-12-15T18:55:05Z)
• Dense Uncertainty Estimation via an Ensemble-based Conditional Latent Variable Model [68.34559610536614]
  We argue that the aleatoric uncertainty is an inherent attribute of the data and can only be correctly estimated with an unbiased oracle model. We propose a new sampling and selection strategy at train time to approximate the oracle model for aleatoric uncertainty estimation. Our results show that our solution achieves both accurate deterministic results and reliable uncertainty estimation.
  arXiv  Detail & Related papers  (2021-11-22T08:54:10Z)
• Counterfactual Evaluation for Explainable AI [21.055319253405603]
  We propose a new methodology to evaluate the faithfulness of explanations from the textitcounterfactual reasoning perspective. We introduce two algorithms to find the proper counterfactuals in both discrete and continuous scenarios and then use the acquired counterfactuals to measure faithfulness.
  arXiv  Detail & Related papers  (2021-09-05T01:38:49Z)
• Temporal Reasoning on Implicit Events from Distant Supervision [91.20159064951487]
  We propose a novel temporal reasoning dataset that evaluates the degree to which systems understand implicit events. We find that state-of-the-art models struggle when predicting temporal relationships between implicit and explicit events. We propose a neuro-symbolic temporal reasoning model, SYMTIME, which exploits distant supervision signals from large-scale text and uses temporal rules to infer end times.
  arXiv  Detail & Related papers  (2020-10-24T03:12:27Z)
• Understanding Neural Abstractive Summarization Models via Uncertainty [54.37665950633147]
  seq2seq abstractive summarization models generate text in a free-form manner. We study the entropy, or uncertainty, of the model's token-level predictions. We show that uncertainty is a useful perspective for analyzing summarization and text generation models more broadly.
  arXiv  Detail & Related papers  (2020-10-15T16:57:27Z)
• Getting a CLUE: A Method for Explaining Uncertainty Estimates [30.367995696223726]
  We propose a novel method for interpreting uncertainty estimates from differentiable probabilistic models. Our method, Counterfactual Latent Uncertainty Explanations (CLUE), indicates how to change an input, while keeping it on the data manifold.
  arXiv  Detail & Related papers  (2020-06-11T21:53:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.