Sketch and Customize: A Counterfactual Story Generator

• URL: http://arxiv.org/abs/2104.00929v1
• Date: Fri, 2 Apr 2021 08:14:22 GMT
• Title: Sketch and Customize: A Counterfactual Story Generator
• Authors: Changying Hao, Liang Pang, Yanyan Lan, Yan Wang, Jiafeng Guo, Xueqi Cheng
• Abstract summary: We propose a sketch-and-customize generation model guided by the causality implicated in the conditions and endings. Experimental results show that the proposed model generates much better endings, as compared with the traditional sequence-to-sequence model.
• Score: 71.34131541754674
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Recent text generation models are easy to generate relevant and fluent text for the given text, while lack of causal reasoning ability when we change some parts of the given text. Counterfactual story rewriting is a recently proposed task to test the causal reasoning ability for text generation models, which requires a model to predict the corresponding story ending when the condition is modified to a counterfactual one. Previous works have shown that the traditional sequence-to-sequence model cannot well handle this problem, as it often captures some spurious correlations between the original and counterfactual endings, instead of the causal relations between conditions and endings. To address this issue, we propose a sketch-and-customize generation model guided by the causality implicated in the conditions and endings. In the sketch stage, a skeleton is extracted by removing words which are conflict to the counterfactual condition, from the original ending. In the customize stage, a generation model is used to fill proper words in the skeleton under the guidance of the counterfactual condition. In this way, the obtained counterfactual ending is both relevant to the original ending and consistent with the counterfactual condition. Experimental results show that the proposed model generates much better endings, as compared with the traditional sequence-to-sequence model.

Related papers

• Counterfactual Generation from Language Models [64.55296662926919]
  We show that counterfactual reasoning is conceptually distinct from interventions. We propose a framework for generating true string counterfactuals. Our experiments demonstrate that the approach produces meaningful counterfactuals.
  arXiv  Detail & Related papers  (2024-11-11T17:57:30Z)
• Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data [49.73114504515852]
  We show that replacing the original real data by each generation's synthetic data does indeed tend towards model collapse. We demonstrate that accumulating the successive generations of synthetic data alongside the original real data avoids model collapse.
  arXiv  Detail & Related papers  (2024-04-01T18:31:24Z)
• Visual Storytelling with Question-Answer Plans [70.89011289754863]
  We present a novel framework which integrates visual representations with pretrained language models and planning. Our model translates the image sequence into a visual prefix, a sequence of continuous embeddings which language models can interpret. It also leverages a sequence of question-answer pairs as a blueprint plan for selecting salient visual concepts and determining how they should be assembled into a narrative.
  arXiv  Detail & Related papers  (2023-10-08T21:45:34Z)
• Model Criticism for Long-Form Text Generation [113.13900836015122]
  We apply a statistical tool, model criticism in latent space, to evaluate the high-level structure of generated text. We perform experiments on three representative aspects of high-level discourse -- coherence, coreference, and topicality. We find that transformer-based language models are able to capture topical structures but have a harder time maintaining structural coherence or modeling coreference.
  arXiv  Detail & Related papers  (2022-10-16T04:35:58Z)
• Text Generation with Text-Editing Models [78.03750739936956]
  This tutorial provides a comprehensive overview of text-editing models and current state-of-the-art approaches. We discuss challenges related to productionization and how these models can be used to mitigate hallucination and bias.
  arXiv  Detail & Related papers  (2022-06-14T17:58:17Z)
• COINS: Dynamically Generating COntextualized Inference Rules for Narrative Story Completion [16.676036625561057]
  We present COINS, a framework that iteratively reads context sentences, generates contextualized inference rules, encodes them, and guides task-specific output generation. By modularizing inference and sentence generation steps in a recurrent model, we aim to make reasoning steps and their effects on next sentence generation transparent. Our automatic and manual evaluations show that the model generates better story sentences than SOTA baselines, especially in terms of coherence.
  arXiv  Detail & Related papers  (2021-06-04T14:06:33Z)
• Consistency and Coherency Enhanced Story Generation [35.08911595854691]
  We propose a two-stage generation framework to enhance consistency and coherency of generated stories. The first stage is to organize the story outline which depicts the story plots and events, and the second stage is to expand the outline into a complete story. In addition, coreference supervision signals are incorporated to reduce coreference errors and improve the coreference consistency.
  arXiv  Detail & Related papers  (2020-10-17T16:40:37Z)
• Narrative Text Generation with a Latent Discrete Plan [39.71663365273463]
  We propose a deep latent variable model that first samples a sequence of anchor words, one per sentence in the story, as part of its generative process. During training, our model treats the sequence of anchor words as a latent variable and attempts to induce anchoring sequences that help guide generation in an unsupervised fashion. We conduct human evaluations which demonstrate that the stories produced by our model are rated better in comparison with baselines which do not consider story plans.
  arXiv  Detail & Related papers  (2020-10-07T08:45:37Z)
• Modeling Preconditions in Text with a Crowd-sourced Dataset [17.828175478279654]
  This paper introduces PeKo, a crowd-sourced annotation of preconditions between event pairs in newswire. We also introduce two challenge tasks aimed at modeling preconditions. Evaluation on both tasks shows that modeling preconditions is challenging even for today's large language models.
  arXiv  Detail & Related papers  (2020-10-06T01:52:34Z)
• Improving Language Generation with Sentence Coherence Objective [4.997730662279843]
  Existing models are often prone to output paragraphs of texts that gradually diverge from the given prompt. The goal of our project is to improve the coherence and consistency across sentences in a language-generation model.
  arXiv  Detail & Related papers  (2020-09-07T06:10:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.