Decoding Methods for Neural Narrative Generation

• URL: http://arxiv.org/abs/2010.07375v2
• Date: Thu, 8 Jul 2021 17:50:35 GMT
• Title: Decoding Methods for Neural Narrative Generation
• Authors: Alexandra DeLucia, Aaron Mueller, Xiang Lisa Li, Jo\~ao Sedoc
• Abstract summary: Narrative generation is an open-ended NLP task in which a model generates a story given a prompt. We apply and evaluate advances in decoding methods for neural response generation to neural narrative generation.
• Score: 74.37264021226308
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Narrative generation is an open-ended NLP task in which a model generates a story given a prompt. The task is similar to neural response generation for chatbots; however, innovations in response generation are often not applied to narrative generation, despite the similarity between these tasks. We aim to bridge this gap by applying and evaluating advances in decoding methods for neural response generation to neural narrative generation. In particular, we employ GPT-2 and perform ablations across nucleus sampling thresholds and diverse decoding hyperparameters -- specifically, maximum mutual information -- analyzing results over multiple criteria with automatic and human evaluation. We find that (1) nucleus sampling is generally best with thresholds between 0.7 and 0.9; (2) a maximum mutual information objective can improve the quality of generated stories; and (3) established automatic metrics do not correlate well with human judgments of narrative quality on any qualitative metric.

Related papers

• Long-Span Question-Answering: Automatic Question Generation and QA-System Ranking via Side-by-Side Evaluation [65.16137964758612]
  We explore the use of long-context capabilities in large language models to create synthetic reading comprehension data from entire books. Our objective is to test the capabilities of LLMs to analyze, understand, and reason over problems that require a detailed comprehension of long spans of text.
  arXiv  Detail & Related papers  (2024-05-31T20:15:10Z)
• Evaluation Metrics of Language Generation Models for Synthetic Traffic Generation Tasks [22.629816738693254]
  We show that common NLG metrics, like BLEU, are not suitable for evaluating Synthetic Traffic Generation (STG) We propose and evaluate several metrics designed to compare the generated traffic to the distribution of real user texts.
  arXiv  Detail & Related papers  (2023-11-21T11:26:26Z)
• PICK: Polished & Informed Candidate Scoring for Knowledge-Grounded Dialogue Systems [59.1250765143521]
  Current knowledge-grounded dialogue systems often fail to align the generated responses with human-preferred qualities. We propose Polished & Informed Candidate Scoring (PICK), a generation re-scoring framework. We demonstrate the effectiveness of PICK in generating responses that are more faithful while keeping them relevant to the dialogue history.
  arXiv  Detail & Related papers  (2023-09-19T08:27:09Z)
• Dialogue-Contextualized Re-ranking for Medical History-Taking [5.039849340960835]
  We present a two-stage re-ranking approach that helps close the training-inference gap by re-ranking the first-stage question candidates. We find that relative to the expert system, the best performance is achieved by our proposed global re-ranker with a transformer backbone.
  arXiv  Detail & Related papers  (2023-04-04T17:31:32Z)
• Robust Preference Learning for Storytelling via Contrastive Reinforcement Learning [53.92465205531759]
  Controlled automated story generation seeks to generate natural language stories satisfying constraints from natural language critiques or preferences. We train a contrastive bi-encoder model to align stories with human critiques, building a general purpose preference model. We further fine-tune the contrastive reward model using a prompt-learning technique to increase story generation robustness.
  arXiv  Detail & Related papers  (2022-10-14T13:21:33Z)
• Unsupervised Neural Stylistic Text Generation using Transfer learning and Adapters [66.17039929803933]
  We propose a novel transfer learning framework which updates only $0.3%$ of model parameters to learn style specific attributes for response generation. We learn style specific attributes from the PERSONALITY-CAPTIONS dataset.
  arXiv  Detail & Related papers  (2022-10-07T00:09:22Z)
• Objective hearing threshold identification from auditory brainstem response measurements using supervised and self-supervised approaches [1.0627340704073347]
  We develop and compare two methods for automated hearing threshold identification from averaged ABR raw data. We show that both models work well, outperform human threshold detection, and are suitable for fast, reliable, and unbiased hearing threshold detection and quality control.
  arXiv  Detail & Related papers  (2021-12-16T15:24:31Z)
• A Template-guided Hybrid Pointer Network for Knowledge-basedTask-oriented Dialogue Systems [15.654119998970499]
  We propose a template-guided hybrid pointer network for the knowledge-based task-oriented dialogue system. We design a memory pointer network model with a gating mechanism to fully exploit the semantic correlation between the retrieved answers and the ground-truth response.
  arXiv  Detail & Related papers  (2021-06-10T15:49:26Z)
• Cue Me In: Content-Inducing Approaches to Interactive Story Generation [74.09575609958743]
  We focus on the task of interactive story generation, where the user provides the model mid-level sentence abstractions. We present two content-inducing approaches to effectively incorporate this additional information. Experimental results from both automatic and human evaluations show that these methods produce more topically coherent and personalized stories.
  arXiv  Detail & Related papers  (2020-10-20T00:36:15Z)
• Asking Questions the Human Way: Scalable Question-Answer Generation from Text Corpus [23.676748207014903]
  We propose Answer-Clue-Style-aware Question Generation (ACS-QG) It aims at automatically generating high-quality and diverse question-answer pairs from unlabeled text corpus at scale. We can generate 2.8 million quality-assured question-answer pairs from a million sentences found in Wikipedia.
  arXiv  Detail & Related papers  (2020-01-27T05:27:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.