Exploring Story Generation with Multi-task Objectives in Variational Autoencoders

• URL: http://arxiv.org/abs/2111.08133v1
• Date: Mon, 15 Nov 2021 23:07:19 GMT
• Title: Exploring Story Generation with Multi-task Objectives in Variational Autoencoders
• Authors: Zhuohan Xie, Trevor Cohn, Jey Han Lau
• Abstract summary: GPT-2 fails to generate consistent stories and lacks diversity. Current story generation models leverage additional information such as plots or commonsense into GPT-2 to guide the generation process. We explore combining BERT and GPT-2 to build a variational autoencoder (VAE) Our evaluations show our enhanced VAE can provide better quality and diversity trade off, generate less repetitive story content and learn a more informative latent variable.
• Score: 41.89428478049741
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: GPT-2 has been frequently adapted in story generation models as it provides powerful generative capability. However, it still fails to generate consistent stories and lacks diversity. Current story generation models leverage additional information such as plots or commonsense into GPT-2 to guide the generation process. These approaches focus on improving generation quality of stories while our work look at both quality and diversity. We explore combining BERT and GPT-2 to build a variational autoencoder (VAE), and extend it by adding additional objectives to learn global features such as story topic and discourse relations. Our evaluations show our enhanced VAE can provide better quality and diversity trade off, generate less repetitive story content and learn a more informative latent variable.

Related papers

• VOLTA: Improving Generative Diversity by Variational Mutual Information Maximizing Autoencoder [38.35049378875308]
  We introduce VOLTA, a framework that elevates generative diversity by bridging Transformer with VAE. We perform comprehensive experiments with two types of Transformers on six datasets to show that our approach can significantly improve generative diversity while maintaining generative quality.
  arXiv  Detail & Related papers  (2023-07-03T08:45:42Z)
• A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT [63.58711128819828]
  ChatGPT and other Generative AI (GAI) techniques belong to the category of Artificial Intelligence Generated Content (AIGC) The goal of AIGC is to make the content creation process more efficient and accessible, allowing for the production of high-quality content at a faster pace.
  arXiv  Detail & Related papers  (2023-03-07T20:36:13Z)
• The Next Chapter: A Study of Large Language Models in Storytelling [51.338324023617034]
  The application of prompt-based learning with large language models (LLMs) has exhibited remarkable performance in diverse natural language processing (NLP) tasks. This paper conducts a comprehensive investigation, utilizing both automatic and human evaluation, to compare the story generation capacity of LLMs with recent models. The results demonstrate that LLMs generate stories of significantly higher quality compared to other story generation models.
  arXiv  Detail & Related papers  (2023-01-24T02:44:02Z)
• Learning to Diversify for Product Question Generation [68.69526529887607]
  We show how the T5 pre-trained Transformer encoder-decoder model can be fine-tuned for the task. We propose a novel learning-to-diversify (LTD) fine-tuning approach that allows to enrich the language learned by the underlying Transformer model.
  arXiv  Detail & Related papers  (2022-07-06T09:26:41Z)
• TegTok: Augmenting Text Generation via Task-specific and Open-world Knowledge [83.55215993730326]
  We propose augmenting TExt Generation via Task-specific and Open-world Knowledge (TegTok) in a unified framework. Our model selects knowledge entries from two types of knowledge sources through dense retrieval and then injects them into the input encoding and output decoding stages respectively.
  arXiv  Detail & Related papers  (2022-03-16T10:37:59Z)
• Goal-Directed Story Generation: Augmenting Generative Language Models with Reinforcement Learning [7.514717103747824]
  We present two automated techniques grounded in deep reinforcement learning and reward shaping to control the plot of computer-generated stories. The first utilizes proximal policy optimization to fine-tune an existing transformer-based language model to generate text continuations but also be goal-seeking. The second extracts a knowledge graph from the unfolding story, which is used by a policy network with graph attention to select a candidate continuation generated by a language model.
  arXiv  Detail & Related papers  (2021-12-16T03:34:14Z)
• Guiding Generative Language Models for Data Augmentation in Few-Shot Text Classification [59.698811329287174]
  We leverage GPT-2 for generating artificial training instances in order to improve classification performance. Our results show that fine-tuning GPT-2 in a handful of label instances leads to consistent classification improvements.
  arXiv  Detail & Related papers  (2021-11-17T12:10:03Z)
• A Temporal Variational Model for Story Generation [21.99104738567138]
  Recent language models can generate interesting and grammatically correct text in story generation but often lack plot development and long-term coherence. This paper experiments with a latent vector planning approach based on a TD-VAE (Temporal Difference Variational Autoencoder) The results demonstrate strong performance in automatic cloze and swapping evaluations.
  arXiv  Detail & Related papers  (2021-09-14T16:36:12Z)
• Consistency and Coherency Enhanced Story Generation [35.08911595854691]
  We propose a two-stage generation framework to enhance consistency and coherency of generated stories. The first stage is to organize the story outline which depicts the story plots and events, and the second stage is to expand the outline into a complete story. In addition, coreference supervision signals are incorporated to reduce coreference errors and improve the coreference consistency.
  arXiv  Detail & Related papers  (2020-10-17T16:40:37Z)
• GenAug: Data Augmentation for Finetuning Text Generators [21.96895115572357]
  We propose and evaluate various augmentation methods, including some that incorporate external knowledge, for finetuning GPT-2 on a subset of Yelp Reviews. Our experiments demonstrate that insertion of character-level synthetic noise and keyword replacement with hypernyms are effective augmentation methods.
  arXiv  Detail & Related papers  (2020-10-05T05:46:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.